Daniel Povey
c92d9d72aa
Fix inf issue
2022-06-10 11:20:47 +08:00
Daniel Povey
950cd4a3e8
Introduce normalization..
2022-06-10 10:47:18 +08:00
Daniel Povey
e891a65735
Move Decorrelate module to after encoder, with scale 0.02->0.1
2022-06-10 10:15:28 +08:00
Daniel Povey
4a5143e548
Increase decay to 1k
2022-06-10 10:09:46 +08:00
Daniel Povey
e2ef8732d1
Increase beta to 0.95
2022-06-10 10:05:28 +08:00
Daniel Povey
a61e21ac85
Change beta to 0.9
2022-06-09 23:33:05 +08:00
Daniel Povey
2c5ebc065e
Change eps to 1e-20
2022-06-09 23:24:33 +08:00
Daniel Povey
c533f91fa2
Remove one line..
2022-06-09 23:13:16 +08:00
Daniel Povey
0fd2cb141f
Code cleanup and refactoring
2022-06-09 22:54:56 +08:00
Daniel Povey
2621cb7f54
Change beta to 0.8
2022-06-09 20:17:12 +08:00
Daniel Povey
082a890635
Fix apply_prob_decay to 500
2022-06-09 19:20:03 +08:00
Daniel Povey
fca844d80c
Make it really have 2k decay and revert to 0.02 scale
2022-06-09 17:45:11 +08:00
Daniel Povey
e99344f15e
Increase scale to 0.04
2022-06-09 13:24:31 +08:00
Daniel Povey
bfcd288afd
Decrease scale on decorrelate component from 0.02 to 0.01
2022-06-09 12:07:19 +08:00
Daniel Povey
56d6dd55ae
Bug fixes
2022-06-09 12:06:35 +08:00
Daniel Povey
1669e21c0c
Use decorrelation in conformer layers also
2022-06-09 11:31:52 +08:00
Daniel Povey
b9a476c7bb
Remove loss factor from decorr_loss_scale
2022-06-08 20:19:17 +08:00
Daniel Povey
8e56445c70
Try to resolve graph-freed problem
2022-06-08 20:07:35 +08:00
Daniel Povey
46ca1cd4c4
Add Decorrelate module that adds something to gradients in backward pass
2022-06-08 19:44:58 +08:00
Daniel Povey
9fb8645168
Implement JoinDropout
2022-06-08 16:17:42 +08:00
Daniel Povey
e7886d49a9
Bug fix
2022-06-08 11:05:29 +08:00
Daniel Povey
a83bde1372
Simplify implementation as current idea was not working to decorrelate
2022-06-08 10:24:41 +08:00
Daniel Povey
135be1e19c
Change dropout_rate from 0.2 to 0.1; fix logging statement; fix assignment to rand_scales, nonrand_scales to use [:]
2022-06-08 00:42:04 +08:00
Daniel Povey
a6050cb2de
Implement new, more principled but maybe slower version.
2022-06-07 23:38:38 +08:00
Daniel Povey
75c822c7e9
Pre and post-multiply by inv_sqrt_stddev,stddev
2022-06-07 20:32:18 +08:00
Daniel Povey
a270973b69
Add gaussian version of decorrelation
2022-06-07 18:55:48 +08:00
Daniel Povey
5d24489752
Have 2 scales on dropout
2022-06-07 18:31:42 +08:00
Daniel Povey
53ca61db7a
Reduce scale on decorrelation by 5, to 0.01
2022-06-07 17:10:54 +08:00
Daniel Povey
7c6d923d3f
Add decorrelation to joiner
2022-06-07 16:47:54 +08:00
Daniel Povey
cd6b707e2b
Various bug fixes
2022-06-07 16:45:32 +08:00
Daniel Povey
40a0934b4e
Implement GaussProjDrop
2022-06-07 11:51:24 +08:00
Daniel Povey
4352a16f57
Fix bug that relates to modifying U in place
2022-06-06 17:43:15 +08:00
Daniel Povey
31848dcd11
Randomize the projections
2022-06-06 16:07:28 +08:00
Daniel Povey
6fdb356315
Bug fix RE GPU device
2022-06-06 15:40:20 +08:00
Daniel Povey
71e927411a
Implement FixedProjDrop
2022-06-06 15:38:59 +08:00
Daniel Povey
28df3ba43f
Fix bug re half precision
2022-06-05 23:26:59 +08:00
Daniel Povey
d76aedb790
Make it work for half
2022-06-05 23:25:51 +08:00
Daniel Povey
e535887abb
Bug fixes.
2022-06-05 23:24:02 +08:00
Daniel Povey
136ffb0597
Add ProjDrop for axis-independent dropout
2022-06-05 23:00:48 +08:00
fanlu
8a3068ead8
Update decode.py ( #392 )
...
* Update decode.py
fix bug ```TypeError: greedy_search_batch() missing 1 required positional argument: 'encoder_out_lens'```
* fix modified_beam_search
Co-authored-by: fanlu3 <fanlu@jd.com>
2022-06-04 19:08:17 +08:00
Zengwei Yao
148f69d8d9
Update RESULTS.md ( #388 )
...
* update RESULT.md about pruned_transducer_stateless4
* Update RESULT.md
This PR is only to update RESULT.md about pruned_transducer_stateless4.
* set default value of --use-averaged-model to True
* update RESULTS.md and add decode command
* minor fix
* update export.py
* add uploaded files links
* update link
* fix typos
2022-06-04 15:52:35 +08:00
Mingshuang Luo
beab229fd7
[Ready to merge] Pruned_transducer_stateless2 for alimeeting dataset ( #378 )
...
* add pruned-rnnt2 recipe for alimeeting dataset
* update code for merging
* change LilcomHdf5Writer to ChunkedLilcomHdf5Writer
* change for test.yml
* change for test.yml
* change for test.yml
* change for workflow yml
* change for yml
* change for yml
* change for README.md
* change for yml
* solve the conflicts
* solve the conflicts
2022-06-04 13:47:46 +08:00
Fangjun Kuang
fbfc98f1d3
Add streaming Emformer stateless RNN-T. ( #390 )
...
* Add streaming Emformer stateless RNN-T.
* Update results for streaming Emformer.
* Minor fixes.
2022-06-01 14:31:47 +08:00
LIyong.Guo
c4ee2bc0af
[Ready to merge]stateless6: states4 + hubert distillation. ( #387 )
...
* a copy of stateless4 as base
* distillation with hubert
* fix typo
* example usage
* usage
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* fix comment
* add results of 100hours
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* check fairseq and quantization
* a short intro to distillation framework
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* add intro of statless6 in README
* fix type error of dst_manifest_dir
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* make export.py call stateless6/train.py instead of stateless2/train.py
* update results by stateless6
* adjust results format
* fix typo
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2022-05-28 12:37:50 +08:00
Mingshuang Luo
c8c8645081
[Ready to merge] Pruned-transducer-stateless2 recipe for aidatatang_200zh ( #375 )
...
* add pruned-rnnt2 model for aidatatang_200zh
* do some changes
* change for README.md
* do some changes
2022-05-24 23:07:40 +08:00
Ewald Enzinger
8c5722de8c
[egs] Add prefix when reading manifests due to recent lhotse changes ( #382 )
...
* [egs] Add prefix when reading manifests due to recent lhotse changes
* Fix wenetspeech
* Fix style issues
2022-05-23 23:37:35 +08:00
Mingshuang Luo
0e57b30495
[Ready to merge] Pruned Transducer Stateless2 for WenetSpeech (char-based) ( #349 )
...
* add char-based pruned-rnnt2 for wenetspeech
* style check
* style check
* change for export.py
* do some changes
* do some changes
* a small change for .flake8
* solve the conflicts
2022-05-23 17:13:01 +08:00
Fangjun Kuang
2f1e23cde1
Narrower and deeper conformer ( #330 )
...
* Copy files for editing.
* Add random combine from #229 .
* Minor fixes.
* Pass model parameters from the command line.
* Fix warnings.
* Fix warnings.
* Update readme.
* Rename to avoid conflicts.
* Update results.
* Add CI for pruned_transducer_stateless5
* Typo fixes.
* Remove random combiner.
* Update decode.py and train.py to use periodically averaged models.
* Minor fixes.
* Revert to use random combiner.
* Update results.
* Minor fixes.
2022-05-23 14:39:11 +08:00
Mingshuang Luo
ec5a112831
[Ready to merge] Do some coding style checks for the latest files ( #379 )
...
* style check
* do changes for .flake8
* a change for compute_fbank_yesno.py
2022-05-20 19:30:38 +08:00
Daniel Povey
2900ed8f8f
Merge pull request #376 from danpovey/diagnostics_fix
...
Diagnostics fix
2022-05-19 12:51:07 +08:00
