Daniel Povey
8bb44b2944
Change param_pow from 0.6 to 0.7
2022-07-19 05:08:32 +08:00
ezerhouni
608473b4eb
Add RNN-LM rescoring in fast beam search ( #475 )
2022-07-18 16:52:17 +08:00
Mingshuang Luo
aec222e2fe
add compile_lg.py for aishell2 recipe ( #481 )
2022-07-18 14:36:40 +08:00
Daniel Povey
bb1e1e154a
Increasing param_pow to 0.6 and decreasing max_lr_factor from 4 to 3.
2022-07-18 09:06:32 +08:00
Daniel Povey
8db3b48edb
Update parameter dependent part of cov more slowly, plus bug fix.
2022-07-18 05:26:55 +08:00
Daniel Povey
198cf2635c
Reduce param_pow from 0.5 to 0.4.
2022-07-17 15:35:07 +08:00
Daniel Povey
3857a87b47
Merge branch 'merge_refactor_param_cov_norank1_iter_batch_max4.0_pow0.5_fix2r_lrupdate200_2k_ns' into merge2_refactor_max4.0_pow0.5_200_1k_ma3.0
2022-07-17 15:32:43 +08:00
Daniel Povey
a572eb4e33
Reducing final lr_update_period from 2k to 1k
2022-07-17 12:56:02 +08:00
Daniel Povey
f36ebad618
Remove 2/3 StructuredLinear/StructuredConv1d modules, use linear/conv1d
2022-07-17 06:40:19 +08:00
Daniel Povey
7e88e2a0e9
Increase debug freq; add type to diagnostics and increase precision of mean,rms
2022-07-17 06:40:16 +08:00
Daniel Povey
de1fd91435
Adding max_abs=3.0 to ActivationBalancer modules inside feedoforward modules.
2022-07-16 07:19:26 +08:00
Daniel Povey
23e6d2e6d8
Fix to the fix
2022-07-16 06:53:44 +08:00
Daniel Povey
4c8d77d14a
Fix return type
2022-07-15 14:18:07 +08:00
ezerhouni
ffca1ae7fb
[WIP] Rnn-T LM nbest rescoring ( #471 )
2022-07-15 10:32:54 +08:00
Daniel Povey
68c5935691
Fix bug re param_cov freshness, properly.
2022-07-15 08:33:10 +08:00
Daniel Povey
b6ee698278
Make LR update period less frequent later in training; fix bug with param_cov freshness, was too fresh
2022-07-15 07:59:30 +08:00
Yuekai Zhang
c17233eca7
[Ready] [Recipes] add aishell2 ( #465 )
...
* add aishell2
* fix aishell2
* add manifest stats
* update prepare char dict
* fix lint
* setting max duration
* lint
* change context size to 1
* update result
* update hf link
* fix decoding comment
* add more decoding methods
* update result
* change context-size 2 default
2022-07-14 14:46:56 +08:00
Daniel Povey
689441b237
Reduce param_pow from 0.75 to 0.5
2022-07-14 06:08:06 +08:00
Daniel Povey
7f6fe02db9
Fix formula for smoothing (was applying more smoothing than intended, and in the opposite sense to intended), also revert max_rms from 2.0 to 4.0
2022-07-14 06:06:02 +08:00
LIyong.Guo
f8d28f0998
update multi_quantization installation ( #469 )
...
* update multi_quantization installation
* Update egs/librispeech/ASR/pruned_transducer_stateless6/train.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2022-07-13 21:16:45 +08:00
Daniel Povey
4785245e5c
Reduce debug freq
2022-07-13 06:51:23 +08:00
Daniel Povey
d48fe0b99c
Change max rms from 10.0 to 4.0
2022-07-13 05:53:35 +08:00
Zengwei Yao
bc2882ddcc
Simplified memory bank for Emformer ( #440 )
...
* init files
* use average value as memory vector for each chunk
* change tail padding length from right_context_length to chunk_length
* correct the files, ln -> cp
* fix bug in conv_emformer_transducer_stateless2/emformer.py
* fix doc in conv_emformer_transducer_stateless/emformer.py
* refactor init states for stream
* modify .flake8
* fix bug about memory mask when memory_size==0
* add @torch.jit.export for init_states function
* update RESULTS.md
* minor change
* update README.md
* modify doc
* replace torch.div() with <<
* fix bug, >> -> <<
* use i&i-1 to judge if it is a power of 2
* minor fix
* fix error in RESULTS.md
2022-07-12 19:19:58 +08:00
Daniel Povey
cedfb5a377
Make max eig ratio 10
2022-07-12 13:59:58 +08:00
Daniel Povey
278358bb9f
Remove debug code
2022-07-12 08:39:14 +08:00
Daniel Povey
8c44ff26f7
Fix bug in batching code for scalars
2022-07-12 08:36:45 +08:00
Daniel Povey
25cb8308d5
Add max_block_size=512 to PrAdam
2022-07-12 08:35:14 +08:00
Daniel Povey
41df045773
Simplify formula, getting rid of scalar_exp_avg_sq
2022-07-11 17:14:12 -07:00
Daniel Povey
4f0e219523
Bug fix to reproduce past results with max_block_size unset.
2022-07-11 17:03:32 -07:00
Daniel Povey
075a2e27d8
Replace max_fullcov_size with max_block_size
2022-07-11 16:37:01 -07:00
Daniel Povey
3468c3aa5a
Remove ActivationBalancer, unnecessary
2022-07-11 14:12:24 -07:00
Daniel Povey
7993c84cd6
Apparently working version, with changed test-code topology
2022-07-11 13:17:29 -07:00
Zengwei Yao
ce26495238
Rand combine update result ( #467 )
...
* update RESULTS.md
* fix test code in pruned_transducer_stateless5/conformer.py
* minor fix
* delete doc
* fix style
2022-07-11 18:13:31 +08:00
Daniel Povey
245d39b1bb
Still debugging but close to done
2022-07-11 00:33:37 -07:00
Daniel Povey
27da50a1f6
Committing partial work..
2022-07-10 15:46:32 -07:00
Daniel Povey
d25df4af5e
Slight refactoring, preparing for batching.
2022-07-09 22:24:36 -07:00
Daniel Povey
d9a6180ae0
Bug fix
2022-07-10 10:20:39 +08:00
Daniel Povey
b7035844a2
Introduce scalar_max, stop eps getting large or small
2022-07-10 10:13:55 +08:00
Daniel Povey
2f73434541
Reduce debug frequency
2022-07-10 06:44:50 +08:00
Daniel Povey
b3bb2dac6f
Iterative, more principled way of estimating param_cov
2022-07-10 06:28:01 +08:00
Daniel Povey
d139c18f22
Max eig of Q limited to 5 times the mean
2022-07-09 14:30:03 +08:00
Daniel Povey
ffeef4ede4
Remove rank-1 dims, meaning where size==numel(), from processing.
2022-07-09 13:36:48 +08:00
Daniel Povey
2fc9eb9789
Respect param_pow
2022-07-09 12:49:04 +08:00
Daniel Povey
209acaf6e4
Increase lr_update_period to 200. The update takes about 2 minutes, fore entire model.
2022-07-09 11:36:54 +08:00
Daniel Povey
61cab3ab65
introduce grad_cov_period
2022-07-09 10:29:23 +08:00
Daniel Povey
35a51bc153
Reduce debug probs
2022-07-09 10:22:19 +08:00
Daniel Povey
65bc964854
Fix bug for scalar update
2022-07-09 10:14:20 +08:00
Daniel Povey
aa2237a793
Bug fix
2022-07-09 10:11:54 +08:00
Daniel Povey
50ee414486
Fix train.py for new optimizer
2022-07-09 10:09:53 +08:00
Daniel Povey
6810849058
Implement new version of learning method. Does more complete diagonalization of grads than the previous methods.
2022-07-09 10:02:17 +08:00
