Daniel Povey
435b073979
Change init of biases to all -0.1..0.1
2022-05-22 10:43:06 +08:00
Daniel Povey
dcab1aee4e
make biases learn faster in a different way.
2022-05-21 17:47:30 +08:00
Daniel Povey
992f922f81
Make biases learn at twice the speed
2022-05-21 17:30:45 +08:00
Daniel Povey
b7adb6d738
Slow down learning of simple_am_proj.weight by 0.5
2022-05-21 17:06:24 +08:00
Daniel Povey
11eac9089e
Fix w.r.t. how cov stats are estimated: storing more stats, and now always zeroing the stats.
2022-05-20 23:05:05 +08:00
Daniel Povey
6f974b32f6
Restore missing factor 1-beta1
2022-05-20 17:43:48 +08:00
Daniel Povey
768c260a4d
Slightly simplify scaling code
2022-05-20 16:43:05 +08:00
Daniel Povey
4257c1024f
Remove initial_speed option
2022-05-20 13:41:56 +08:00
Daniel Povey
abe5abb688
Implement Cain with scdaling incorporated;
...
Removing scaling from ScaledLinear, ScaledConv1d, etc.
2022-05-20 13:36:15 +08:00
Daniel Povey
8fd9e64fdf
Fix eps value
2022-05-19 22:25:18 +08:00
Daniel Povey
1edc0fa841
Fixes to cain
2022-05-19 22:21:41 +08:00
Daniel Povey
6085ab64ef
Make cain average over more iters and use preconditioning on the other dims first
2022-05-19 21:34:12 +08:00
Daniel Povey
e44edf99a4
Merge branch 'master' into knowledge_base_1b_L2_ng_orth2feat_cain
2022-05-19 12:42:50 +08:00
Daniel Povey
c736b39c7d
Remove unnecessary option for diagnostics code, collect on more batches
2022-05-19 11:35:54 +08:00
Daniel Povey
ebc2ffeff7
Bug fix
2022-05-18 10:26:29 +08:00
Daniel Povey
668d01cc7a
Replace Eve with Cain in pruned_transducer_stateless4
2022-05-18 10:16:50 +08:00
Fangjun Kuang
f6ce135608
Various fixes to support torch script. ( #371 )
...
* Various fixes to support torch script.
* Add tests to ensure that the model is torch scriptable.
* Update tests.
2022-05-16 21:46:59 +08:00
Daniel Povey
995371ad95
Move train.py changes to the right dir
2022-05-15 22:17:08 +08:00
Daniel Povey
cee5396058
Orthogonalize every 2k iters
2022-05-15 21:50:40 +08:00
Daniel Povey
0989aec741
Replace Eve optimizer with Abel.
2022-05-15 14:33:08 +08:00
Fangjun Kuang
aeb8986e35
Ignore padding frames during RNN-T decoding. ( #358 )
...
* Ignore padding frames during RNN-T decoding.
* Fix outdated decoding code.
* Minor fixes.
2022-05-13 07:39:14 +08:00
Zengwei Yao
20f092e709
Support decoding with averaged model when using --iter ( #353 )
...
* support decoding with averaged model when using --iter
* minor fix
* monir fix of copyright date
2022-05-07 13:09:11 +08:00
Zengwei Yao
c059ef3169
Keep model_avg on cpu ( #348 )
...
* keep model_avg on cpu
* explicitly convert model_avg to cpu
* minor fix
* remove device convertion for model_avg
* modify usage of the model device in train.py
* change model.device to next(model.parameters()).device for decoding
* assert params.start_epoch>0
* assert params.start_epoch>0, params.start_epoch
2022-05-07 10:42:34 +08:00
Zengwei Yao
00c48ec1f3
Model average ( #344 )
...
* First upload of model average codes.
* minor fix
* update decode file
* update .flake8
* rename pruned_transducer_stateless3 to pruned_transducer_stateless4
* change epoch number counter starting from 1 instead of 0
* minor fix of pruned_transducer_stateless4/train.py
* refactor the checkpoint.py
* minor fix, update docs, and modify the epoch number to count from 1 in the pruned_transducer_stateless4/decode.py
* update author info
* add docs of the scaling in function average_checkpoints_with_averaged_model
2022-05-05 21:20:04 +08:00
