Daniel Povey
8e454bcf9e
Exclude size=500 dim from projection; try to use double for model average
2022-05-26 15:15:27 +08:00
Mingshuang Luo
c8c8645081
[Ready to merge] Pruned-transducer-stateless2 recipe for aidatatang_200zh ( #375 )
...
* add pruned-rnnt2 model for aidatatang_200zh
* do some changes
* change for README.md
* do some changes
2022-05-24 23:07:40 +08:00
Ewald Enzinger
8c5722de8c
[egs] Add prefix when reading manifests due to recent lhotse changes ( #382 )
...
* [egs] Add prefix when reading manifests due to recent lhotse changes
* Fix wenetspeech
* Fix style issues
2022-05-23 23:37:35 +08:00
Mingshuang Luo
0e57b30495
[Ready to merge] Pruned Transducer Stateless2 for WenetSpeech (char-based) ( #349 )
...
* add char-based pruned-rnnt2 for wenetspeech
* style check
* style check
* change for export.py
* do some changes
* do some changes
* a small change for .flake8
* solve the conflicts
2022-05-23 17:13:01 +08:00
Fangjun Kuang
2f1e23cde1
Narrower and deeper conformer ( #330 )
...
* Copy files for editing.
* Add random combine from #229 .
* Minor fixes.
* Pass model parameters from the command line.
* Fix warnings.
* Fix warnings.
* Update readme.
* Rename to avoid conflicts.
* Update results.
* Add CI for pruned_transducer_stateless5
* Typo fixes.
* Remove random combiner.
* Update decode.py and train.py to use periodically averaged models.
* Minor fixes.
* Revert to use random combiner.
* Update results.
* Minor fixes.
2022-05-23 14:39:11 +08:00
Daniel Povey
9ef11e64ba
Some small fixes, to bias_correction2 formula and remove bias-u,v-scale
2022-05-22 16:28:33 +08:00
Daniel Povey
b916789ca3
Further increase scales
2022-05-22 12:25:26 +08:00
Daniel Povey
9e206d53fc
Increase initial scale for conv and self_attn
2022-05-22 12:18:57 +08:00
Daniel Povey
56d9928934
Scale down modules at initialization
2022-05-22 11:56:59 +08:00
Daniel Povey
5d57dd3930
Change initial bias scales from 0.1 to 0.2
2022-05-22 10:59:51 +08:00
Daniel Povey
435b073979
Change init of biases to all -0.1..0.1
2022-05-22 10:43:06 +08:00
Daniel Povey
cbdddbe670
Reduce scale on bias from 4 to 2
2022-05-21 20:07:41 +08:00
Daniel Povey
dcab1aee4e
make biases learn faster in a different way.
2022-05-21 17:47:30 +08:00
Daniel Povey
992f922f81
Make biases learn at twice the speed
2022-05-21 17:30:45 +08:00
Daniel Povey
b7adb6d738
Slow down learning of simple_am_proj.weight by 0.5
2022-05-21 17:06:24 +08:00
Daniel Povey
11eac9089e
Fix w.r.t. how cov stats are estimated: storing more stats, and now always zeroing the stats.
2022-05-20 23:05:05 +08:00
Mingshuang Luo
ec5a112831
[Ready to merge] Do some coding style checks for the latest files ( #379 )
...
* style check
* do changes for .flake8
* a change for compute_fbank_yesno.py
2022-05-20 19:30:38 +08:00
Daniel Povey
6f974b32f6
Restore missing factor 1-beta1
2022-05-20 17:43:48 +08:00
Daniel Povey
768c260a4d
Slightly simplify scaling code
2022-05-20 16:43:05 +08:00
Daniel Povey
4257c1024f
Remove initial_speed option
2022-05-20 13:41:56 +08:00
Daniel Povey
abe5abb688
Implement Cain with scdaling incorporated;
...
Removing scaling from ScaledLinear, ScaledConv1d, etc.
2022-05-20 13:36:15 +08:00
Daniel Povey
8fd9e64fdf
Fix eps value
2022-05-19 22:25:18 +08:00
Daniel Povey
1edc0fa841
Fixes to cain
2022-05-19 22:21:41 +08:00
Daniel Povey
6085ab64ef
Make cain average over more iters and use preconditioning on the other dims first
2022-05-19 21:34:12 +08:00
Daniel Povey
2900ed8f8f
Merge pull request #376 from danpovey/diagnostics_fix
...
Diagnostics fix
2022-05-19 12:51:07 +08:00
Daniel Povey
ac5a9faafd
Merge branch 'master' into knowledge_base_1b_L2_ng_orth2feat_cain
2022-05-19 12:49:24 +08:00
Daniel Povey
9e88d0bf31
Merge remote-tracking branch 'upstream/master'
2022-05-19 12:49:12 +08:00
Daniel Povey
5230e73e41
Small fixes
2022-05-19 12:49:00 +08:00
Daniel Povey
e44edf99a4
Merge branch 'master' into knowledge_base_1b_L2_ng_orth2feat_cain
2022-05-19 12:42:50 +08:00
Daniel Povey
4e23fb2252
Improve diagnostics code memory-wise and accumulate more stats. ( #373 )
...
* Update diagnostics, hopefully print more stats.
# Conflicts:
# egs/librispeech/ASR/pruned_transducer_stateless4b/train.py
* Remove memory-limit options arg
* Remove unnecessary option for diagnostics code, collect on more batches
2022-05-19 11:45:59 +08:00
Daniel Povey
c736b39c7d
Remove unnecessary option for diagnostics code, collect on more batches
2022-05-19 11:35:54 +08:00
Daniel Povey
c0fdfabaf3
Remove memory-limit options arg
2022-05-19 11:30:56 +08:00
Daniel Povey
c2c46ea023
Update diagnostics, hopefully print more stats.
...
# Conflicts:
# egs/librispeech/ASR/pruned_transducer_stateless4b/train.py
2022-05-19 11:29:31 +08:00
Daniel Povey
ebc2ffeff7
Bug fix
2022-05-18 10:26:29 +08:00
Daniel Povey
668d01cc7a
Replace Eve with Cain in pruned_transducer_stateless4
2022-05-18 10:16:50 +08:00
Daniel Povey
38ccbb6062
Think it works now.. will move to different branch though.
2022-05-18 10:11:20 +08:00
Daniel Povey
3d35018ce0
Just adding some draft code, will move this around.
2022-05-17 21:00:36 +08:00
Daniel Povey
a46f74feb4
Broadcast transformation
2022-05-17 17:23:28 +08:00
Daniel Povey
ceb4eb4b85
Scale covariance
2022-05-17 15:32:06 +08:00
Daniel Povey
c923b5900e
Symmetrize covariance
2022-05-17 15:14:23 +08:00
Daniel Povey
9133b57808
Use torch.no_grad() for stats
2022-05-17 14:49:00 +08:00
Daniel Povey
07d3369234
Finish feat-diagonalizing code
2022-05-17 14:13:56 +08:00
Fangjun Kuang
f6ce135608
Various fixes to support torch script. ( #371 )
...
* Various fixes to support torch script.
* Add tests to ensure that the model is torch scriptable.
* Update tests.
2022-05-16 21:46:59 +08:00
Desh Raj
5aafbb970e
SPGISpeech recipe ( #334 )
...
* initial commit for SPGISpeech recipe
* add decoding
* add spgispeech transducer
* remove conformer ctc; minor fixes in RNN-T
* add results
* add tensorboard
* add pretrained model to HF
* remove unused scripts and soft link common scripts
* remove duplicate files
* pre commit hooks
* remove change in librispeech
* pre commit hook
* add CER numbers
2022-05-16 20:52:14 +08:00
Daniel Povey
2f2934a115
fix train.py
2022-05-16 19:41:16 +08:00
Daniel Povey
d61c8aa3bc
Hopefully this finishes the full orthogonalization.
2022-05-16 19:18:12 +08:00
Daniel Povey
67f916e599
Draft towards 2nd orthogonalization
2022-05-16 16:16:12 +08:00
Daniel Povey
8aeaf1421a
Rename orthogonalize to diagonalize
2022-05-16 12:43:47 +08:00
Daniel Povey
9859e33c06
Remove optimizer.reset() which is not supported for Eve
2022-05-16 12:13:36 +08:00
Daniel Povey
d58debdb3f
Use Eve with orthogonalization
2022-05-16 10:34:25 +08:00
