mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-09-07 16:14:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
520 Commits 5 Branches 3 Tags
Commit Graph

520 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Povey
9e206d53fc Increase initial scale for conv and self_attn 2022-05-22 12:18:57 +08:00
Daniel Povey
56d9928934 Scale down modules at initialization 2022-05-22 11:56:59 +08:00
Daniel Povey
5d57dd3930 Change initial bias scales from 0.1 to 0.2 2022-05-22 10:59:51 +08:00
Daniel Povey
435b073979 Change init of biases to all -0.1..0.1 2022-05-22 10:43:06 +08:00
Daniel Povey
cbdddbe670 Reduce scale on bias from 4 to 2 2022-05-21 20:07:41 +08:00
Daniel Povey
dcab1aee4e make biases learn faster in a different way. 2022-05-21 17:47:30 +08:00
Daniel Povey
992f922f81 Make biases learn at twice the speed 2022-05-21 17:30:45 +08:00
Daniel Povey
b7adb6d738 Slow down learning of simple_am_proj.weight by 0.5 2022-05-21 17:06:24 +08:00
Daniel Povey
11eac9089e Fix w.r.t. how cov stats are estimated: storing more stats, and now always zeroing the stats. 2022-05-20 23:05:05 +08:00
Daniel Povey
6f974b32f6 Restore missing factor 1-beta1 2022-05-20 17:43:48 +08:00
Daniel Povey
768c260a4d Slightly simplify scaling code 2022-05-20 16:43:05 +08:00
Daniel Povey
4257c1024f Remove initial_speed option 2022-05-20 13:41:56 +08:00
Daniel Povey
abe5abb688 Implement Cain with scdaling incorporated; 
Removing scaling from ScaledLinear, ScaledConv1d, etc.
 2022-05-20 13:36:15 +08:00
Daniel Povey
8fd9e64fdf Fix eps value 2022-05-19 22:25:18 +08:00
Daniel Povey
1edc0fa841 Fixes to cain 2022-05-19 22:21:41 +08:00
Daniel Povey
6085ab64ef Make cain average over more iters and use preconditioning on the other dims first 2022-05-19 21:34:12 +08:00
Daniel Povey
ac5a9faafd Merge branch 'master' into knowledge_base_1b_L2_ng_orth2feat_cain 2022-05-19 12:49:24 +08:00
Daniel Povey
9e88d0bf31 Merge remote-tracking branch 'upstream/master' 2022-05-19 12:49:12 +08:00
Daniel Povey
5230e73e41 Small fixes 2022-05-19 12:49:00 +08:00
Daniel Povey
e44edf99a4 Merge branch 'master' into knowledge_base_1b_L2_ng_orth2feat_cain 2022-05-19 12:42:50 +08:00
Daniel Povey
4e23fb2252
Improve diagnostics code memory-wise and accumulate more stats. (#373) 
* Update diagnostics, hopefully print more stats.

# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless4b/train.py

* Remove memory-limit options arg

* Remove unnecessary option for diagnostics code, collect on more batches
 2022-05-19 11:45:59 +08:00
Daniel Povey
c736b39c7d Remove unnecessary option for diagnostics code, collect on more batches 2022-05-19 11:35:54 +08:00
Daniel Povey
c0fdfabaf3 Remove memory-limit options arg 2022-05-19 11:30:56 +08:00
Daniel Povey
c2c46ea023 Update diagnostics, hopefully print more stats. 
# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless4b/train.py
 2022-05-19 11:29:31 +08:00
Daniel Povey
ebc2ffeff7 Bug fix 2022-05-18 10:26:29 +08:00
Daniel Povey
668d01cc7a Replace Eve with Cain in pruned_transducer_stateless4 2022-05-18 10:16:50 +08:00
Daniel Povey
38ccbb6062 Think it works now.. will move to different branch though. 2022-05-18 10:11:20 +08:00
Daniel Povey
3d35018ce0 Just adding some draft code, will move this around. 2022-05-17 21:00:36 +08:00
Daniel Povey
a46f74feb4 Broadcast transformation 2022-05-17 17:23:28 +08:00
Daniel Povey
ceb4eb4b85 Scale covariance 2022-05-17 15:32:06 +08:00
Daniel Povey
c923b5900e Symmetrize covariance 2022-05-17 15:14:23 +08:00
Daniel Povey
9133b57808 Use torch.no_grad() for stats 2022-05-17 14:49:00 +08:00
Daniel Povey
07d3369234 Finish feat-diagonalizing code 2022-05-17 14:13:56 +08:00
Fangjun Kuang
f6ce135608
Various fixes to support torch script. (#371) 
* Various fixes to support torch script.

* Add tests to ensure that the model is torch scriptable.

* Update tests.
 2022-05-16 21:46:59 +08:00
Desh Raj
5aafbb970e
SPGISpeech recipe (#334) 
* initial commit for SPGISpeech recipe

* add decoding

* add spgispeech transducer

* remove conformer ctc; minor fixes in RNN-T

* add results

* add tensorboard

* add pretrained model to HF

* remove unused scripts and soft link common scripts

* remove duplicate files

* pre commit hooks

* remove change in librispeech

* pre commit hook

* add CER numbers
 2022-05-16 20:52:14 +08:00
Daniel Povey
2f2934a115 fix train.py 2022-05-16 19:41:16 +08:00
Daniel Povey
d61c8aa3bc Hopefully this finishes the full orthogonalization. 2022-05-16 19:18:12 +08:00
Daniel Povey
67f916e599 Draft towards 2nd orthogonalization 2022-05-16 16:16:12 +08:00
Daniel Povey
8aeaf1421a Rename orthogonalize to diagonalize 2022-05-16 12:43:47 +08:00
Daniel Povey
9859e33c06 Remove optimizer.reset() which is not supported for Eve 2022-05-16 12:13:36 +08:00
Daniel Povey
d58debdb3f Use Eve with orthogonalization 2022-05-16 10:34:25 +08:00
Daniel Povey
995371ad95 Move train.py changes to the right dir 2022-05-15 22:17:08 +08:00
Daniel Povey
cee5396058 Orthogonalize every 2k iters 2022-05-15 21:50:40 +08:00
Daniel Povey
bb32556f9e Add and test reset() function 2022-05-15 16:20:10 +08:00
Daniel Povey
a1dc020270 train scalars slower 2022-05-15 15:44:56 +08:00
Daniel Povey
0989aec741 Replace Eve optimizer with Abel. 2022-05-15 14:33:08 +08:00
Fangjun Kuang
6f7860a0a6
Fix GitHub CI for decoding GigaSpeech dev/test datasets (#366) 2022-05-15 14:25:35 +08:00
Daniel Povey
5e3bf4ce5a Remove some debug code 2022-05-15 14:22:12 +08:00
Daniel Povey
6306f24430 Some changes to algorithm; more diagnostics printing 2022-05-15 14:12:23 +08:00
Daniel Povey
67c402a369 Add some debugging/diagnostic code 2022-05-15 13:28:00 +08:00