Yifan Yang
d667dc365b
Fix for diagnostic ( #1135 )
...
* CTC loss return tensor
* Update model.py
2023-06-16 15:04:41 +08:00
Yifan Yang
0a465794a8
Fix Zipformer ( #1132 )
...
* Update model.py
* Update train.py
* Update decoder.py
2023-06-15 17:52:14 +08:00
Fangjun Kuang
947f0614c9
Fix running exported model on GPU. ( #1131 )
2023-06-15 12:25:15 +08:00
Zengwei Yao
0ad037d076
Add CTC loss option in zipformer recipe ( #1111 )
...
* add CTC loss option in zipformer recipe
* add ctc_decode.py
* support CTC model export, add jit_pretrained_ctc.py, pretrained_ctc.py
* update README.md and RESULTS.md
* add CI test
2023-06-14 14:27:29 +08:00
danfu
0cb71ad3bc
add updated zipformer onnx export ( #1108 )
...
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-12 14:02:23 +08:00
Peter Ross
b4c38d7547
Use symlinks for best epochs ( #1123 )
...
* utils: add symlink_or_copyfile
* pruned_transducer_stateless7: use symlinks (when possible) to output best epochs
* Rename function
---------
Co-authored-by: Yifan Yang <64255737+yfyeung@users.noreply.github.com>
2023-06-12 13:51:46 +08:00
Yifan Yang
dca21c2a17
Fix parameters_names in train.py ( #1121 )
2023-06-08 16:54:05 +08:00
SarahSmitho
3ae47a4940
verify have installed ffmpeg ( #1117 )
2023-06-07 11:17:38 +08:00
Fangjun Kuang
c0de78d3c0
Add data preparation for the MuST-C speech translation corpus ( #1107 )
2023-06-05 15:49:41 +08:00
Wei Kang
ba257efbcd
Add Context biasing ( #1038 )
...
* Add context biasing for librispeech
* Add context biasing for wenetspeech
* fix bugs
* Implement Aho-Corasick context graph
* fix some bugs
* Fixes to forward_one_step; add draw to context graph
* add output arc; fix black
* Fix wenetspeech tokenizer
* Minor fixes to the decode.py
2023-06-03 21:28:49 +08:00
Yifan Yang
ca60ced213
Fix typo ( #1114 )
...
* Fix typo for zipformer
* Fix typo for pruned_transducer_stateless7
* Fix typo for pruned_transducer_stateless7_ctc
* Fix typo for pruned_transducer_stateless7_ctc_bs
* Fix typo for pruned_transducer_stateless7_streaming
* Fix typo for pruned_transducer_stateless7_streaming_multi
* Fix file permissions for pruned_transducer_stateless7_streaming_multi
* Fix typo for pruned_transducer_stateless8
* Fix typo for pruned_transducer_stateless6
* Fix typo for pruned_transducer_stateless5
* Fix typo for pruned_transducer_stateless4
* Fix typo for pruned_transducer_stateless3
2023-06-02 14:12:42 +08:00
Yifan Yang
82f34a2388
Remove multidataset from librispeech/pruned_transducer_stateless7 ( #1105 )
...
* Add People's Speech to multidataset
* update
* remove multi from librispeech
2023-06-01 18:45:20 +08:00
Yifan Yang
03853f1ee5
Add peoples_speech ( #1101 )
...
* update
* Small fix
* Update egs/peoples_speech/ASR/prepare.sh
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* limit normalize log
* Update egs/peoples_speech/ASR/local/compute_fbank_peoples_speech_valid_test.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* Update compute_fbank_peoples_speech_splits.py
* Update compute_fbank_peoples_speech_valid_test.py
---------
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-05-31 12:46:17 +08:00
Fangjun Kuang
7b0afbdc16
Remove cur_batch_idx ( #1102 )
2023-05-30 14:49:54 +08:00
Fangjun Kuang
1aeffa73bc
remove outdated code in train.py ( #1096 )
2023-05-25 07:47:38 +08:00
Zengwei Yao
6826b076d4
add flops profiler, support for Zipformer encoder and Conformer encoder ( #1093 )
...
* add flops profiler, support for Zipformer encoder and Conformer encoder
* support for reworked conformer and old zipformer
* skip black check
2023-05-24 19:10:45 +08:00
Fangjun Kuang
1df71a6b38
add onnx export for stateless2 ( #1086 )
2023-05-23 16:11:00 +08:00
Fangjun Kuang
ea8b15309f
Add onnx export scripts for wenetspeech recipe. ( #1085 )
2023-05-23 13:32:14 +08:00
Fangjun Kuang
dbcf0b41db
Fix stateless7 training error ( #1082 )
2023-05-23 12:52:02 +08:00
marcoyang1998
585e7b224f
Aishell pruned_transducer_stateless7 ( #962 )
...
* Add pruned_transducer_stateless7 for Aishell
* update README.md
* update comments and small fixes
2023-05-23 11:04:33 +08:00
Yifan Yang
90c392b7b3
Add docs for Fine-tune with mux ( #1074 )
...
* Update RESULTS.md
2023-05-22 12:39:51 +08:00
Zengwei Yao
8070258ec5
fix conv_emformer2, when using right_context_length=0 ( #1076 )
2023-05-21 20:31:54 +08:00
Zengwei Yao
30fcd16c7d
rm zipformer/__init__.py ( #1075 )
2023-05-20 23:12:11 +08:00
Zengwei Yao
a7e142b7ff
Support long audios recognition ( #980 )
...
* support long file transcription
* rename recipe as long_file_recog
* add docs
* support multi-gpu decoding
* style fix
2023-05-19 20:27:55 +08:00
Zengwei Yao
f18b539fbc
Add the upgraded Zipformer model ( #1058 )
...
* add the zipformer codes, copied from branch from_dan_scaled_adam_exp1119
* support model export with torch.jit.script
* update RESULTS.md
* support exporting streaming model with torch.jit.script
* add results of streaming models, with some minor changes
* update README.md
* add CI test
* update k2 version in requirements-ci.txt
* update pyproject.toml
2023-05-19 16:47:59 +08:00
Fangjun Kuang
ae1949ddcc
Support using the latest master from tencent/ncnn ( #1070 )
...
* Support using the latest master from tencent/ncnn
* small fixes
2023-05-18 20:56:58 +08:00
Yifan Yang
562bda91e4
Add adaption recipe for pruned_transducer_stateless7 ( #1059 )
...
* Add mux for finetune
* Add comments
* Fix for black
* Update finetune.py
2023-05-17 16:02:27 +08:00
Wei Kang
bccd20d978
Traning with byte level BPE (TAL_CSASR) ( #1033 )
...
* Add byte level bpe tal_csasr recipe
* Minor fixes to decoding and exporting
* Fix prepare.sh
* Update results
2023-05-16 12:44:52 +08:00
Fangjun Kuang
6c326427a0
Support exporting streaming conformer to ONNX ( #1047 )
2023-05-10 14:47:37 +08:00
Fangjun Kuang
5b50ffda54
support using mini librispeech in training ( #1048 )
...
* support mini librispeech in training
* update onnx export doc
2023-05-09 15:10:06 +08:00
Fangjun Kuang
ebbab37776
Fix broken code in download_lm.py ( #1046 )
2023-05-08 20:48:17 +08:00
Peter Ross
62c9dd9703
make egs/timit work according to the documentation ( #1044 )
...
* prepare.sh: restore working directory after git lfs pull
* set execute permisons on python scripts called by prepare.sh
2023-05-08 19:07:40 +08:00
Fangjun Kuang
efbb577b88
fix compiling HLG ( #1039 )
2023-05-07 16:26:13 +08:00
Yifan Yang
98569b2607
Update RESULTS.md ( #1036 )
...
* Update RESULTS.md
2023-05-06 17:51:55 +08:00
Wei Kang
80156dda09
Training with byte level BPE (AIShell) ( #986 )
...
* copy files from zipformer librispeech
* Add byte bpe training for aishell
* compile LG graph
* Support LG decoding
* Minor fixes
* black
* Minor fixes
* export & fix pretrain.py
* fix black
* Update RESULTS.md
* Fix export.py
2023-05-04 19:16:17 +08:00
Yuanhang Zhang
b0228c536e
Fix typo in librispeech OpenFST-based HLG preparation script ( #1028 )
2023-04-28 19:52:32 +08:00
marcoyang1998
45c13e90e4
RNNLM rescore + Low-order density ratio ( #1017 )
...
* add rnnlm rescore + LODR
* add LODR in decode.py
* update RESULTS
2023-04-24 15:00:02 +08:00
Yifan Yang
2096e69bda
Use CutSet.mux for multidataset ( #1020 )
...
* Use CutSet.mux
* Remove mischange
* Fix for style check
2023-04-23 18:41:44 +08:00
Yifan Yang
d67a49afe4
Add multidataset ( #1010 )
...
* Add Common Voice for multidataset
* Add prepare_multidataset.sh
* Add dataset mixing
* Update prepare_multidataset.sh
* Update prepare_giga_speech.sh
* update comments
* Add split and shuffle mechanism
* Add multi-dataset train
* Fix for deleting
* Fix for modifying
* Add comments
* Change type for perturb_speed
* Fix for style check
* Small fix
* Add filter
* Remove warning
2023-04-21 18:09:41 +08:00
marcoyang1998
57d6482a79
Streaming Zipformer with multi-dataset ( #984 )
...
* modify train.py
* add right padding option in decode.py
* update RESULTS.md
2023-04-21 15:43:28 +08:00
Wei Kang
0efed1cec5
Fix path in aishell rnnlm training ( #1016 )
2023-04-20 23:09:31 +08:00
Wei Kang
5c65516e05
Fix aishell rnnlm training command ( #1015 )
2023-04-20 16:14:16 +08:00
Yifan Yang
81d386ef3e
Add compute_ppl.py and ngram_entropy_pruning.py ( #1013 )
2023-04-20 12:27:43 +08:00
Wen Ding
78b9dcc936
Support exporting BS Zipformer models to ONNX, used in Triton Server ( #1008 )
...
* Support export BS Zipformer models to ONNX in Tritron
* Update copyright
* Update exporting codes for BS zipformer models
* Code format
* Update comments
* Update export_onnx.py
---------
Co-authored-by: Yifan Yang <64255737+yfyeung@users.noreply.github.com>
2023-04-18 17:05:08 +08:00
Yifan Yang
05e7435d0d
Move soft links into proper position ( #1007 )
2023-04-18 10:11:12 +08:00
Yifan Yang
8838fe0bd2
Zipformer for Common Voice ( #997 )
...
* Add soft links in pruned_transducer_stateless7 for CommonVoice
* Add python files
* Update prepare.sh
* Update normalization
* Fix for soft links
* Add some docs
* Add export
* Update egs/commonvoice/ASR/RESULTS.md
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
* Add export for onnx
---------
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-04-17 17:47:25 +08:00
marcoyang1998
34d1b07c3d
Modified beam search with RNNLM rescoring ( #1002 )
...
* add RNNLM rescore
* add shallow fusion and lm rescore for streaming zipformer
* minor fix
* update RESULTS.md
* fix yesno workflow, change from ubuntu-18.04 to ubuntu-latest
2023-04-17 16:43:00 +08:00
Fangjun Kuang
e32658e620
Fix torch.jit.script() export for streaming zipformer. ( #1005 )
2023-04-17 16:13:30 +08:00
Zengwei Yao
7c7d9ab042
add @torch.jit.export for streaming_forward func in Zipformer class ( #1004 )
2023-04-17 12:03:52 +08:00
Zengwei Yao
5f066d3d53
support decoding and computing RTF on test sets with onnx models ( #995 )
...
* support decode and compute RTF on test sets with onnx models
* support onnx export and decode in pruned_transducer_stateless
2023-04-12 19:04:50 +08:00
