Daniel Povey
c7cf229f56
Revers pruned_transducer_stateless4 to upstream/master
2022-05-31 12:45:51 +08:00
Daniel Povey
b01c09a693
Remove the natural gradient stuff while keeping cosmetic changes.
2022-05-30 11:56:11 +08:00
Daniel Povey
2b8ea98fc2
Improve documentation; remove unused code.
2022-05-28 19:22:09 +08:00
Daniel Povey
295595d334
Revert the exclusion of dim=500
2022-05-28 17:49:16 +08:00
Daniel Povey
7aa47408af
Bug fixes to avoid inf alpha
2022-05-27 21:41:05 +08:00
Daniel Povey
fd0e9d4bad
Fix bug for scalars.
2022-05-27 20:48:41 +08:00
Daniel Povey
503b79252f
Add new update and max_eff_lr
2022-05-27 20:44:10 +08:00
Daniel Povey
4efe920401
More consistent use of eps.
2022-05-27 17:41:19 +08:00
Daniel Povey
eed864a3db
Change power from 0.66 to 1.0, like natural gradient.
2022-05-27 16:45:42 +08:00
Daniel Povey
89fad8cc5a
Change power to 0.66
2022-05-27 16:39:54 +08:00
Daniel Povey
8e454bcf9e
Exclude size=500 dim from projection; try to use double for model average
2022-05-26 15:15:27 +08:00
Daniel Povey
9ef11e64ba
Some small fixes, to bias_correction2 formula and remove bias-u,v-scale
2022-05-22 16:28:33 +08:00
Daniel Povey
11eac9089e
Fix w.r.t. how cov stats are estimated: storing more stats, and now always zeroing the stats.
2022-05-20 23:05:05 +08:00
Daniel Povey
6f974b32f6
Restore missing factor 1-beta1
2022-05-20 17:43:48 +08:00
Daniel Povey
768c260a4d
Slightly simplify scaling code
2022-05-20 16:43:05 +08:00
Daniel Povey
abe5abb688
Implement Cain with scdaling incorporated;
...
Removing scaling from ScaledLinear, ScaledConv1d, etc.
2022-05-20 13:36:15 +08:00
Daniel Povey
8fd9e64fdf
Fix eps value
2022-05-19 22:25:18 +08:00
Daniel Povey
1edc0fa841
Fixes to cain
2022-05-19 22:21:41 +08:00
Daniel Povey
6085ab64ef
Make cain average over more iters and use preconditioning on the other dims first
2022-05-19 21:34:12 +08:00
Daniel Povey
ebc2ffeff7
Bug fix
2022-05-18 10:26:29 +08:00
Daniel Povey
668d01cc7a
Replace Eve with Cain in pruned_transducer_stateless4
2022-05-18 10:16:50 +08:00
Zengwei Yao
00c48ec1f3
Model average ( #344 )
...
* First upload of model average codes.
* minor fix
* update decode file
* update .flake8
* rename pruned_transducer_stateless3 to pruned_transducer_stateless4
* change epoch number counter starting from 1 instead of 0
* minor fix of pruned_transducer_stateless4/train.py
* refactor the checkpoint.py
* minor fix, update docs, and modify the epoch number to count from 1 in the pruned_transducer_stateless4/decode.py
* update author info
* add docs of the scaling in function average_checkpoints_with_averaged_model
2022-05-05 21:20:04 +08:00
