LIyong.Guo
c4ee2bc0af
[Ready to merge]stateless6: states4 + hubert distillation. ( #387 )
...
* a copy of stateless4 as base
* distillation with hubert
* fix typo
* example usage
* usage
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* fix comment
* add results of 100hours
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* check fairseq and quantization
* a short intro to distillation framework
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* add intro of statless6 in README
* fix type error of dst_manifest_dir
* Update egs/librispeech/ASR/pruned_transducer_stateless6/hubert_xlarge.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* make export.py call stateless6/train.py instead of stateless2/train.py
* update results by stateless6
* adjust results format
* fix typo
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2022-05-28 12:37:50 +08:00
yaozengwei
545316636b
Merge remote-tracking branch 'origin/master'
2022-05-26 21:55:56 +08:00
yaozengwei
fbbc24f941
Merge remote-tracking branch 'k2-fsa/master'
2022-05-26 21:54:40 +08:00
Mingshuang Luo
c8c8645081
[Ready to merge] Pruned-transducer-stateless2 recipe for aidatatang_200zh ( #375 )
...
* add pruned-rnnt2 model for aidatatang_200zh
* do some changes
* change for README.md
* do some changes
2022-05-24 23:07:40 +08:00
Ewald Enzinger
8c5722de8c
[egs] Add prefix when reading manifests due to recent lhotse changes ( #382 )
...
* [egs] Add prefix when reading manifests due to recent lhotse changes
* Fix wenetspeech
* Fix style issues
2022-05-23 23:37:35 +08:00
Mingshuang Luo
0e57b30495
[Ready to merge] Pruned Transducer Stateless2 for WenetSpeech (char-based) ( #349 )
...
* add char-based pruned-rnnt2 for wenetspeech
* style check
* style check
* change for export.py
* do some changes
* do some changes
* a small change for .flake8
* solve the conflicts
2022-05-23 17:13:01 +08:00
Fangjun Kuang
2f1e23cde1
Narrower and deeper conformer ( #330 )
...
* Copy files for editing.
* Add random combine from #229 .
* Minor fixes.
* Pass model parameters from the command line.
* Fix warnings.
* Fix warnings.
* Update readme.
* Rename to avoid conflicts.
* Update results.
* Add CI for pruned_transducer_stateless5
* Typo fixes.
* Remove random combiner.
* Update decode.py and train.py to use periodically averaged models.
* Minor fixes.
* Revert to use random combiner.
* Update results.
* Minor fixes.
2022-05-23 14:39:11 +08:00
Mingshuang Luo
ec5a112831
[Ready to merge] Do some coding style checks for the latest files ( #379 )
...
* style check
* do changes for .flake8
* a change for compute_fbank_yesno.py
2022-05-20 19:30:38 +08:00
Daniel Povey
2900ed8f8f
Merge pull request #376 from danpovey/diagnostics_fix
...
Diagnostics fix
2022-05-19 12:51:07 +08:00
Daniel Povey
9e88d0bf31
Merge remote-tracking branch 'upstream/master'
2022-05-19 12:49:12 +08:00
Daniel Povey
5230e73e41
Small fixes
2022-05-19 12:49:00 +08:00
Daniel Povey
4e23fb2252
Improve diagnostics code memory-wise and accumulate more stats. ( #373 )
...
* Update diagnostics, hopefully print more stats.
# Conflicts:
# egs/librispeech/ASR/pruned_transducer_stateless4b/train.py
* Remove memory-limit options arg
* Remove unnecessary option for diagnostics code, collect on more batches
2022-05-19 11:45:59 +08:00
Daniel Povey
c736b39c7d
Remove unnecessary option for diagnostics code, collect on more batches
2022-05-19 11:35:54 +08:00
Daniel Povey
c0fdfabaf3
Remove memory-limit options arg
2022-05-19 11:30:56 +08:00
Daniel Povey
c2c46ea023
Update diagnostics, hopefully print more stats.
...
# Conflicts:
# egs/librispeech/ASR/pruned_transducer_stateless4b/train.py
2022-05-19 11:29:31 +08:00
Fangjun Kuang
f6ce135608
Various fixes to support torch script. ( #371 )
...
* Various fixes to support torch script.
* Add tests to ensure that the model is torch scriptable.
* Update tests.
2022-05-16 21:46:59 +08:00
Desh Raj
5aafbb970e
SPGISpeech recipe ( #334 )
...
* initial commit for SPGISpeech recipe
* add decoding
* add spgispeech transducer
* remove conformer ctc; minor fixes in RNN-T
* add results
* add tensorboard
* add pretrained model to HF
* remove unused scripts and soft link common scripts
* remove duplicate files
* pre commit hooks
* remove change in librispeech
* pre commit hook
* add CER numbers
2022-05-16 20:52:14 +08:00
yaozengwei
c9d84aeb5c
Merge remote-tracking branch 'k2-fsa/master'
2022-05-15 18:02:27 +08:00
Fangjun Kuang
6f7860a0a6
Fix GitHub CI for decoding GigaSpeech dev/test datasets ( #366 )
2022-05-15 14:25:35 +08:00
Guanbo Wang
9630f9a3ba
Update GigaSpeech reults ( #364 )
...
* Update decode.py
* Update export.py
* Update results
* Update README.md
2022-05-15 12:57:40 +08:00
Fangjun Kuang
f23dd43719
Update results for libri+giga multi dataset setup. ( #363 )
...
* Update results for libri+giga multi dataset setup.
2022-05-14 21:45:39 +08:00
Fangjun Kuang
2d7096dfc6
Decode gigaspeech in GitHub actions ( #362 )
...
* Add CI for gigaspeech.
2022-05-14 08:53:22 +08:00
Fangjun Kuang
0f180b3ce2
Validate that there are no OOV tokens in BPE-based lexicons. ( #359 )
...
* Validate that there are no OOV tokens in BPE-based lexicons.
* Typo fixes.
2022-05-13 14:00:35 +08:00
Fangjun Kuang
e30e042c39
Update decoding script for gigaspeech and remove duplicate files. ( #361 )
2022-05-13 13:03:16 +08:00
Guanbo Wang
48a6a9a549
GigaSpeech RNN-T experiments ( #318 )
...
* Copy RNN-T recipe from librispeech
* flake8
* flake8
* Update params
* gigaspeech decode
* black
* Update results
* syntax highlight
* Update RESULTS.md
* typo
2022-05-13 11:03:26 +08:00
Fangjun Kuang
7b7acdf369
Support --iter in export.py ( #360 )
2022-05-13 10:51:44 +08:00
Fangjun Kuang
aeb8986e35
Ignore padding frames during RNN-T decoding. ( #358 )
...
* Ignore padding frames during RNN-T decoding.
* Fix outdated decoding code.
* Minor fixes.
2022-05-13 07:39:14 +08:00
yaozengwei
bcef517a84
Merge remote-tracking branch 'k2-fsa/master'
2022-05-12 17:45:45 +08:00
Fangjun Kuang
bc284e88e6
Run decode.py in GitHub actions. ( #356 )
2022-05-10 14:51:34 +08:00
Fangjun Kuang
cd460f7bf1
Stringify torch.__version__ before serializing it. ( #354 )
2022-05-07 17:18:34 +08:00
Zengwei Yao
20f092e709
Support decoding with averaged model when using --iter ( #353 )
...
* support decoding with averaged model when using --iter
* minor fix
* monir fix of copyright date
2022-05-07 13:09:11 +08:00
Mingshuang Luo
f783e10dc8
Do some changes for aishell/ASR/transducer stateless/export.py ( #347 )
...
* do some changes for aishell/ASR/transducer_stateless/export.py
2022-05-07 11:09:31 +08:00
yaozengwei
ecfb3e9c26
Merge remote-tracking branch 'k2-fsa/master'
2022-05-07 11:07:48 +08:00
Zengwei Yao
c059ef3169
Keep model_avg on cpu ( #348 )
...
* keep model_avg on cpu
* explicitly convert model_avg to cpu
* minor fix
* remove device convertion for model_avg
* modify usage of the model device in train.py
* change model.device to next(model.parameters()).device for decoding
* assert params.start_epoch>0
* assert params.start_epoch>0, params.start_epoch
2022-05-07 10:42:34 +08:00
Guanbo Wang
8e3c89076e
Bug fix ( #352 )
2022-05-07 08:10:54 +08:00
Fangjun Kuang
32f05c00e3
Save batch to disk on exception. ( #350 )
2022-05-06 17:49:40 +08:00
yaozengwei
70634d58a1
Merge remote-tracking branch 'k2-fsa/master'
2022-05-06 11:31:20 +08:00
Zengwei Yao
00c48ec1f3
Model average ( #344 )
...
* First upload of model average codes.
* minor fix
* update decode file
* update .flake8
* rename pruned_transducer_stateless3 to pruned_transducer_stateless4
* change epoch number counter starting from 1 instead of 0
* minor fix of pruned_transducer_stateless4/train.py
* refactor the checkpoint.py
* minor fix, update docs, and modify the epoch number to count from 1 in the pruned_transducer_stateless4/decode.py
* update author info
* add docs of the scaling in function average_checkpoints_with_averaged_model
2022-05-05 21:20:04 +08:00
Fangjun Kuang
8635fb4334
Fix decoding for gigaspeech in the libri + giga setup. ( #345 )
2022-05-05 20:58:46 +08:00
Fangjun Kuang
e1c3e98980
Save batch to disk on OOM. ( #343 )
...
* Save batch to disk on OOM.
* minor fixes
* Fixes after review.
* Fix style issues.
2022-05-05 15:09:23 +08:00
Fangjun Kuang
9ddbc681e7
Validate generated manifest files. ( #338 )
2022-05-03 07:08:33 +08:00
Fangjun Kuang
6af15914fa
Validate generated manifest files. ( #338 )
2022-05-03 07:02:54 +08:00
Fangjun Kuang
6dc2e04462
Update results. ( #340 )
...
* Update results.
* Typo fixes.
2022-04-29 15:49:45 +08:00
Fangjun Kuang
ac84220de9
Modified conformer with multi datasets ( #312 )
...
* Copy files for editing.
* Use librispeech + gigaspeech with modified conformer.
* Support specifying number of workers for on-the-fly feature extraction.
* Feature extraction code for GigaSpeech.
* Combine XL splits lazily during training.
* Fix warnings in decoding.
* Add decoding code for GigaSpeech.
* Fix decoding the gigaspeech dataset.
We have to use the decoder/joiner networks for the GigaSpeech dataset.
* Disable speed perturbe for XL subset.
* Compute the Nbest oracle WER for RNN-T decoding.
* Minor fixes.
* Minor fixes.
* Add results.
* Update results.
* Update CI.
* Update results.
* Fix style issues.
* Update results.
* Fix style issues.
2022-04-29 15:40:30 +08:00
yaozengwei
9c39d8b009
Merge remote-tracking branch 'k2-fsa/master'
2022-04-29 10:26:06 +08:00
Fangjun Kuang
caab6cfd92
Support specifying iteration number of checkpoints for decoding. ( #336 )
...
See also #289
2022-04-28 14:09:22 +08:00
Fangjun Kuang
9aeea3e1af
Support averaging models with weight tying. ( #333 )
2022-04-26 13:32:03 +08:00
pehonnet
9a98e6ced6
fix fp16 option in example usage ( #332 )
2022-04-25 18:51:53 +08:00
whsqkaak
d766dc5aee
Fix some typos. ( #329 )
2022-04-22 15:54:59 +08:00
Fangjun Kuang
3607c516d6
Update results for torchaudio RNN-T. ( #322 )
2022-04-20 11:15:10 +08:00
