zr_jin
a81396b482
Use tokens.txt to replace bpe.model ( #1162 )
2023-08-12 16:53:59 +08:00
Fangjun Kuang
1dbbd7759e
Add tests for subsample.py and fix typos ( #1180 )
2023-07-25 14:46:18 +08:00
Yifan Yang
ca60ced213
Fix typo ( #1114 )
...
* Fix typo for zipformer
* Fix typo for pruned_transducer_stateless7
* Fix typo for pruned_transducer_stateless7_ctc
* Fix typo for pruned_transducer_stateless7_ctc_bs
* Fix typo for pruned_transducer_stateless7_streaming
* Fix typo for pruned_transducer_stateless7_streaming_multi
* Fix file permissions for pruned_transducer_stateless7_streaming_multi
* Fix typo for pruned_transducer_stateless8
* Fix typo for pruned_transducer_stateless6
* Fix typo for pruned_transducer_stateless5
* Fix typo for pruned_transducer_stateless4
* Fix typo for pruned_transducer_stateless3
2023-06-02 14:12:42 +08:00
Yifan Yang
d67a49afe4
Add multidataset ( #1010 )
...
* Add Common Voice for multidataset
* Add prepare_multidataset.sh
* Add dataset mixing
* Update prepare_multidataset.sh
* Update prepare_giga_speech.sh
* update comments
* Add split and shuffle mechanism
* Add multi-dataset train
* Fix for deleting
* Fix for modifying
* Add comments
* Change type for perturb_speed
* Fix for style check
* Small fix
* Add filter
* Remove warning
2023-04-21 18:09:41 +08:00
Zengwei Yao
5f066d3d53
support decoding and computing RTF on test sets with onnx models ( #995 )
...
* support decode and compute RTF on test sets with onnx models
* support onnx export and decode in pruned_transducer_stateless
2023-04-12 19:04:50 +08:00
Zengwei Yao
136aa94d57
remove duplicated lines ( #988 )
2023-04-06 17:47:33 +08:00
marcoyang1998
d337398d29
Shallow fusion for Aishell ( #954 )
...
* add shallow fusion and LODR for aishell
* update RESULTS
* add save by iterations
2023-04-03 16:20:29 +08:00
Fangjun Kuang
a632b24c35
Export int8 quantized models for non-streaming Zipformer. ( #977 )
...
* Export int8 quantized models for non-streaming Zipformer.
* Delete export-onnx.py
* Export int8 models for other folders
2023-03-31 22:46:19 +08:00
Zengwei Yao
2a5a75cb56
add option of using full attention for streaming model decoding ( #975 )
2023-03-30 14:30:13 +08:00
Fangjun Kuang
8c3ea93fc8
Save meta data to exported ONNX models ( #968 )
2023-03-27 11:39:29 +08:00
Fangjun Kuang
f5de2e90c6
Fix style issues. ( #937 )
2023-03-08 22:56:04 +08:00
pehonnet
07243d136a
remove key from result filename ( #936 )
...
Co-authored-by: pe-honnet <pe.honnet@telepathy.ai>
2023-03-08 21:06:07 +08:00
Fangjun Kuang
2b995639b7
Add ONNX support for Zipformer and ConvEmformer ( #884 )
2023-02-09 00:02:38 +08:00
Fangjun Kuang
7ae03f6c88
Add onnx export support for pruned_transducer_stateless5 ( #883 )
2023-02-07 17:47:08 +08:00
Fangjun Kuang
8d3810e289
Simplify ONNX export ( #881 )
...
* Simplify ONNX export
* Fix ONNX CI tests
2023-02-07 15:01:59 +08:00
marcoyang1998
1f0408b103
Support Transformer LM ( #750 )
...
* support transformer LM
* show number of parameters during training
* update docstring
* testing files for ppl calculation
* add lm wrampper for rnn and transformer LM
* apply lm wrapper in lm shallow fusion
* small updates
* update decode.py to support LM fusion and LODR
* add export.py
* update CI and workflow
* update decoding results
* fix CI
* remove transformer LM from CI test
2022-12-29 10:53:36 +08:00
Fangjun Kuang
88b7895adf
fix librispeech.py in multi-dataset setup ( #791 )
2022-12-27 13:59:55 +08:00
Daniil
b293db4baf
Tedlium3 conformer ctc2 ( #696 )
...
* modify preparation
* small refacor
* add tedlium3 conformer_ctc2
* modify decode
* filter unk in decode
* add scaling converter
* address comments
* fix lambda function lhotse
* add implicit manifest shuffle
* refactor ctc_greedy_search
* import model arguments from train.py
* style fix
* fix ci test and last style issues
* update RESULTS
* fix RESULTS numbers
* fix label smoothing loss
* update model parameters number in RESULTS
2022-12-13 16:13:26 +08:00
Fangjun Kuang
6533f359c9
Fix CI ( #726 )
...
* Fix CI
* Disable shuffle for yesno.
See https://github.com/k2-fsa/icefall/issues/197
2022-12-02 10:53:06 +08:00
marcoyang1998
4b5bc480e8
Add low-order density ratio in RNNLM shallow fusion ( #678 )
...
* Support LODR in RNNLM shallow fusion
* fix style
* fix code style
* update workflow and CI
* update results
* propagate changes to stateless3
* add decoding results for stateless3+giga
* fix CI
2022-11-30 17:26:05 +08:00
huangruizhe
6693d907d3
shuffle full Librispeech data ( #574 )
...
* shuffled full/partial librispeech data
* fixed the code style issue
* Shuffled full librispeech data off-line
* Fixed style, addressed comments, and removed redandunt codes
* Used the suggested version of black
* Propagated the changes to other folders for librispeech (except
conformer_mmi and streaming_conformer_ctc)
2022-11-27 11:26:09 +08:00
Desh Raj
d31db01037
manual correction of black formatting
2022-11-17 14:18:05 -05:00
Desh Raj
107df3b115
apply black on all files
2022-11-17 09:42:17 -05:00
Fangjun Kuang
60317120ca
Revert "Apply new Black style changes"
2022-11-17 20:19:32 +08:00
Desh Raj
d110b04ad3
apply new black formatting to all files
2022-11-16 13:06:43 -05:00
Tiance Wang
952a7b3fcc
Fix typo ( #681 )
...
* Update add_alignment_librispeech.py
* Update scaling_converter.py
2022-11-15 10:45:48 +08:00
Fangjun Kuang
e334e570d8
Filter utterances with number_tokens > number_feature_frames. ( #604 )
2022-11-12 07:57:58 +08:00
Zengwei Yao
3600ce1b5f
Apply delay penalty on transducer ( #654 )
...
* add delay penalty
* fix CI
* fix CI
2022-11-04 16:10:09 +08:00
Wei Kang
64aed2cdeb
Fix LG log file name ( #657 )
2022-11-03 23:12:35 +08:00
Wei Kang
163d929601
Add fast_beam_search_LG ( #622 )
...
* Add fast_beam_search_LG
* add fast_beam_search_LG to commonly used recipes
* fix ci
* fix ci
* Fix error
2022-11-03 16:29:30 +08:00
Wei Kang
d389524d45
remove tail padding for non-streaming models ( #625 )
2022-11-01 11:09:56 +08:00
Fangjun Kuang
7f1c0e07b6
Remove onnx and onnxruntime from requirements.txt ( #640 )
...
* Remove onnx and onnxruntime from requirements.txt
2022-10-31 13:44:40 +08:00
Wei Kang
581d0361cc
Fix type hints for decode.py ( #638 )
...
* Fix type hints for decode.py
* Fix flake8
2022-10-30 16:35:30 +08:00
Nagendra Goel
6709bf1e63
Update train.py ( #635 )
...
Add the missing step to add the arguments to the parser.
2022-10-28 10:23:32 +08:00
Fangjun Kuang
d69bb826ed
Support exporting LSTM with projection to ONNX ( #621 )
...
* Support exporting LSTM with projection to ONNX
* Add missing files
* small fixes
2022-10-18 11:25:31 +08:00
Fangjun Kuang
d1f16a04bd
fix type hints for decode.py ( #623 )
2022-10-18 06:56:12 +08:00
Fangjun Kuang
a66e74b92f
Fix links in the doc ( #619 )
2022-10-14 12:23:47 +08:00
Fangjun Kuang
c39cba5191
Support exporting to ONNX for the wenetspeech recipe ( #615 )
...
* Support exporting to ONNX for the wenetspeech recipe
2022-10-13 15:17:20 +08:00
Zengwei Yao
aa58c2ee02
Modify ActivationBalancer for speed ( #612 )
...
* add a probability to apply ActivationBalancer
* minor fix
* minor fix
2022-10-13 15:14:28 +08:00
Fangjun Kuang
1c07d2fb37
Remove all-in-one for onnx export ( #614 )
...
* Remove all-in-one for onnx export
* Exit on error for CI
2022-10-12 10:34:06 +08:00
Yunusemre
f3db4ea871
exporting projection layers of joiner separately for onnx ( #584 )
...
* exporting projection layers of joiner separately for onnx
2022-10-11 18:22:28 +08:00
Fangjun Kuang
099cd3a215
support exporting to ncnn format via PNNX ( #571 )
2022-09-20 22:52:49 +08:00
Fangjun Kuang
97b3fc53aa
Add LSTM for the multi-dataset setup. ( #558 )
...
* Add LSTM for the multi-dataset setup.
* Add results
* fix style issues
* add missing file
2022-09-16 18:40:25 +08:00
kobenaxie
235eb0746f
fix scaling converter test for decoder(predictor). ( #553 )
2022-08-27 17:26:21 +08:00
Yuekai Zhang
f9c3d7f92f
fix typo for export jit script ( #544 )
2022-08-23 17:29:42 +08:00
Zengwei Yao
f2f5baf687
Use ScaledLSTM as streaming encoder ( #479 )
...
* add ScaledLSTM
* add RNNEncoderLayer and RNNEncoder classes in lstm.py
* add RNN and Conv2dSubsampling classes in lstm.py
* hardcode bidirectional=False
* link from pruned_transducer_stateless2
* link scaling.py pruned_transducer_stateless2
* copy from pruned_transducer_stateless2
* modify decode.py pretrained.py test_model.py train.py
* copy streaming decoding files from pruned_transducer_stateless2
* modify streaming decoding files
* simplified code in ScaledLSTM
* flat weights after scaling
* pruned2 -> pruned4
* link __init__.py
* fix style
* remove add_model_arguments
* modify .flake8
* fix style
* fix scale value in scaling.py
* add random combiner for training deeper model
* add using proj_size
* add scaling converter for ScaledLSTM
* support jit trace
* add using averaged model in export.py
* modify test_model.py, test if the model can be successfully exported by jit.trace
* modify pretrained.py
* support streaming decoding
* fix model.py
* Add cut_id to recognition results
* Add cut_id to recognition results
* do not pad in Conv subsampling module; add tail padding during decoding.
* update RESULTS.md
* minor fix
* fix doc
* update README.md
* minor change, filter infinite loss
* remove the condition of raise error
* modify type hint for the return value in model.py
* minor change
* modify RESULTS.md
Co-authored-by: pkufool <wkang.pku@gmail.com>
2022-08-19 14:38:45 +08:00
marcoyang1998
c74cec59e9
propagate changes from #525 to other librispeech recipes ( #531 )
...
* propaga changes from #525 to other librispeech recipes
* refactor display_and_save_batch to utils
* fixed typo
* reformat code style
2022-08-17 17:18:15 +08:00
Wei Kang
5c17255eec
Sort results to make it more convenient to compare decoding results ( #522 )
...
* Sort result to make it more convenient to compare decoding results
* Add cut_id to recognition results
* add cut_id to results for all recipes
* Fix torch.jit.script
* Fix comments
* Minor fixes
* Fix torch.jit.tracing for Pytorch version before v1.9.0
2022-08-12 07:12:50 +08:00
Fangjun Kuang
1f7832b93c
Fix loading sampler state dict. ( #421 )
...
* Fix loading sampler state dict.
* skip scan_pessimistic_batches_for_oom if params.start_batch > 0
2022-08-06 10:00:08 +08:00
Yunusemre
7157f62af3
Merging onnx models ( #518 )
...
* add export function of onnx-all-in-one to export.py
* add onnx_check script for all-in-one onnx model
* minor fix
* remove unused arguments
* add onnx-all-in-one test
* fix style
* fix style
* fix requirements
* fix input/output names
* fix installing onnx_graphsurgeon
* fix instaliing onnx_graphsurgeon
* revert to previous requirements.txt
* fix minor
2022-08-04 23:03:41 +08:00
