Daniel Povey
e597b68160
Improve roundoff properties of algorithm for getting C, use randomized testing.
2022-06-14 12:59:41 +08:00
Daniel Povey
d729e970e1
Some changes for stability
2022-06-14 10:25:17 +08:00
yaozengwei
ec8646d0cd
Merge remote-tracking branch 'k2-fsa/master'
2022-06-13 20:55:28 +08:00
Daniel Povey
d1e96afce2
Improve speed by messing with configuration, removing asserts.
2022-06-13 17:27:34 +08:00
Daniel Povey
a41c4b6c9b
Bug fixes
2022-06-13 17:20:46 +08:00
Daniel Povey
41b0e582df
Bug fix to typo in optim.py
2022-06-13 16:10:49 +08:00
Daniel Povey
ca7cffcb42
Remove Decorrelate() class
2022-06-13 16:08:32 +08:00
Daniel Povey
7338c60296
Remove Decorrelate()
2022-06-13 16:07:15 +08:00
Daniel Povey
c1f487e36d
Move optim2.py to optim.py; use this optimizer in train.py
2022-06-13 16:05:46 +08:00
Daniel Povey
4e0d8c45f4
Move optim2.py to optim.py; use this optimizer in train.py
2022-06-13 16:05:22 +08:00
Zengwei Yao
53f38c01d2
Emformer with conv module and scaling mechanism ( #389 )
...
* copy files from existing branch
* add rule in .flake8
* monir style fix
* fix typos
* add tail padding
* refactor, use fixed-length cache for batch decoding
* copy from streaming branch
* copy from streaming branch
* modify emformer states stack and unstack, streaming decoding, to be continued
* refactor Stream class
* remane streaming_feature_extractor.py
* refactor streaming decoding
* test states stack and unstack
* fix bugs, no grad, and num_proccessed_frames
* add modify_beam_search, fast_beam_search
* support torch.jit.export
* use torch.div
* copy from pruned_transducer_stateless4
* modify export.py
* add author info
* delete other test functions
* minor fix
* modify doc
* fix style
* minor fix doc
* minor fix
* minor fix doc
* update RESULTS.md
* fix typo
* add info
* fix typo
* fix doc
* add test function for conv module, and minor fix.
* add copyright info
* minor change of test_emformer.py
* fix doc of stack and unstack, test case with batch_size=1
* update README.md
2022-06-13 15:09:17 +08:00
Daniel Povey
d5b90f278e
Adding draft of new optimizer
2022-06-13 15:01:10 +08:00
yaozengwei
2a5a70e03e
Merge remote-tracking branch 'k2-fsa/master'
2022-06-13 12:52:28 +08:00
Daniel Povey
8c4b5454e7
Remove unnecessary line
2022-06-11 13:14:18 +08:00
Daniel Povey
73fd2fcd77
Updates to cain to new param_rms per element
2022-06-11 13:08:08 +08:00
Daniel Povey
d301f8ac6c
Merge Decorrelate work, and simplification to RandomCombine, into pruned_transducer_stateless7
2022-06-11 11:07:07 +08:00
Daniel Povey
30379bb35d
Merge branch 'cain3d_merge_supersimple' into merge_decorr_cain
2022-06-11 10:51:30 +08:00
Daniel Povey
a3c97554e8
Reduce scale from 0.1 to 0.05
2022-06-10 23:34:46 +08:00
Daniel Povey
2bbc63a2f5
Change first1k to decay1k
2022-06-10 23:33:16 +08:00
Daniel Povey
41cdbae753
Increase scale from 0.05 to 0.1
2022-06-10 19:07:48 +08:00
Daniel Povey
5fb64a59b8
Change beta from 0.8 to 0.95
2022-06-10 19:05:04 +08:00
Daniel Povey
6ed181595b
Scale by grad norm
2022-06-10 18:34:42 +08:00
Daniel Povey
6a47bf1178
Decrease scale to 0.05
2022-06-10 18:33:50 +08:00
Daniel Povey
9d4633facf
Simplified gradient scaling [no scaling]; only use 1k first iters; beta =0.8
2022-06-10 17:16:05 +08:00
Daniel Povey
cecd52155c
Apply x row scaling with grad
2022-06-10 16:42:52 +08:00
Daniel Povey
86c2d0fcc0
Make it run only for first 3k steps; larger scale; remove limit of 1.0
2022-06-10 16:33:22 +08:00
Daniel Povey
eeb95ed502
Fix issue with cov scale
2022-06-10 16:25:45 +08:00
Daniel Povey
c671e213fc
Increase beta from 0.95 to 0.98
2022-06-10 14:39:58 +08:00
Daniel Povey
c6cfb1e5fa
Remove logging that was excessive
2022-06-10 14:25:23 +08:00
Daniel Povey
ff0309947a
Do scaling a different way, so loss function is more consistent; accum stats in backward pass
2022-06-10 14:16:44 +08:00
Daniel Povey
58cbc3d961
Move PseudoNormalizeFunction to a different place.
2022-06-10 14:01:13 +08:00
Daniel Povey
77357cdaa8
Change scale from 0.1 to 0.05
2022-06-10 12:42:43 +08:00
Daniel Povey
173c8e6ca3
Make decorrelate scale really 0.1
2022-06-10 12:39:01 +08:00
Fangjun Kuang
9f6c748b30
Add links to sherpa. ( #417 )
...
* Add links to sherpa.
2022-06-10 12:19:18 +08:00
Fangjun Kuang
bfeab319c9
Fix aishell. ( #416 )
2022-06-10 11:47:43 +08:00
Daniel Povey
42667aacf9
Remove linear layers from RandomCombine
2022-06-10 11:38:50 +08:00
Daniel Povey
c92d9d72aa
Fix inf issue
2022-06-10 11:20:47 +08:00
Daniel Povey
950cd4a3e8
Introduce normalization..
2022-06-10 10:47:18 +08:00
Daniel Povey
e891a65735
Move Decorrelate module to after encoder, with scale 0.02->0.1
2022-06-10 10:15:28 +08:00
Daniel Povey
4a5143e548
Increase decay to 1k
2022-06-10 10:09:46 +08:00
Daniel Povey
e2ef8732d1
Increase beta to 0.95
2022-06-10 10:05:28 +08:00
Daniel Povey
a61e21ac85
Change beta to 0.9
2022-06-09 23:33:05 +08:00
Daniel Povey
2c5ebc065e
Change eps to 1e-20
2022-06-09 23:24:33 +08:00
Daniel Povey
c533f91fa2
Remove one line..
2022-06-09 23:13:16 +08:00
Daniel Povey
0fd2cb141f
Code cleanup and refactoring
2022-06-09 22:54:56 +08:00
Daniel Povey
2621cb7f54
Change beta to 0.8
2022-06-09 20:17:12 +08:00
Daniel Povey
082a890635
Fix apply_prob_decay to 500
2022-06-09 19:20:03 +08:00
Daniel Povey
fca844d80c
Make it really have 2k decay and revert to 0.02 scale
2022-06-09 17:45:11 +08:00
Daniel Povey
e99344f15e
Increase scale to 0.04
2022-06-09 13:24:31 +08:00
Daniel Povey
bfcd288afd
Decrease scale on decorrelate component from 0.02 to 0.01
2022-06-09 12:07:19 +08:00
