870 Commits

Author SHA1 Message Date
marcoyang1998
8be9f0d562
Update docs/source/decoding-with-langugage-models/shallow-fusion.rst
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-29 12:10:00 +08:00
marcoyang1998
b55dd5e364
Update docs/source/decoding-with-langugage-models/LODR.rst
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-29 12:09:53 +08:00
marcoyang1998
c0709c8107
Update docs/source/decoding-with-langugage-models/LODR.rst
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-29 12:09:40 +08:00
marcoyang1998
78fec8ef6f
Update docs/source/decoding-with-langugage-models/LODR.rst
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-29 12:08:12 +08:00
marcoyang1998
5ff647e226
Update docs/source/decoding-with-langugage-models/LODR.rst
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-29 12:08:02 +08:00
marcoyang
ec942c25cf minor fixes 2023-06-29 10:16:15 +08:00
marcoyang
3b3ada765c minor fixes 2023-06-28 17:25:42 +08:00
marcoyang
34682d3b07 minor updates 2023-06-28 17:05:08 +08:00
marcoyang
3207ceab46 update documentation for shallow fusion 2023-06-28 16:53:09 +08:00
marcoyang
2ada280379 add documentation for LODR 2023-06-28 16:52:57 +08:00
marcoyang
8abe24cc77 add LODR 2023-06-25 18:39:29 +08:00
marcoyang
0fbdadfe7b change wording 2023-06-20 17:09:52 +08:00
marcoyang
542bbc936e minor fix 2023-06-20 17:02:32 +08:00
marcoyang
645e2a5ed8 add shallow fusion documentation 2023-06-20 17:02:21 +08:00
marcoyang
ad24b4ad9e resolve conflict 2023-06-19 12:31:35 +08:00
Yifan Yang
d667dc365b
Fix for diagnostic (#1135)
* CTC loss return tensor

* Update model.py
2023-06-16 15:04:41 +08:00
Yifan Yang
0a465794a8
Fix Zipformer (#1132)
* Update model.py

* Update train.py

* Update decoder.py
2023-06-15 17:52:14 +08:00
Fangjun Kuang
947f0614c9
Fix running exported model on GPU. (#1131) 2023-06-15 12:25:15 +08:00
Zengwei Yao
0ad037d076
Add CTC loss option in zipformer recipe (#1111)
* add CTC loss option in zipformer recipe

* add ctc_decode.py

* support CTC model export, add jit_pretrained_ctc.py, pretrained_ctc.py

* update README.md and RESULTS.md

* add CI test
2023-06-14 14:27:29 +08:00
danfu
0cb71ad3bc
add updated zipformer onnx export (#1108)
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-12 14:02:23 +08:00
Peter Ross
b4c38d7547
Use symlinks for best epochs (#1123)
* utils: add symlink_or_copyfile

* pruned_transducer_stateless7: use symlinks (when possible) to output best epochs

* Rename function

---------

Co-authored-by: Yifan Yang <64255737+yfyeung@users.noreply.github.com>
2023-06-12 13:51:46 +08:00
Yifan Yang
dca21c2a17
Fix parameters_names in train.py (#1121) 2023-06-08 16:54:05 +08:00
SarahSmitho
3ae47a4940
verify have installed ffmpeg (#1117) 2023-06-07 11:17:38 +08:00
Fangjun Kuang
c0de78d3c0
Add data preparation for the MuST-C speech translation corpus (#1107) 2023-06-05 15:49:41 +08:00
Wei Kang
ba257efbcd
Add Context biasing (#1038)
* Add context biasing for librispeech

* Add context biasing for wenetspeech

* fix bugs

* Implement Aho-Corasick context graph

* fix some bugs

* Fixes to forward_one_step; add draw to context graph

* add output arc; fix black

* Fix wenetspeech tokenizer

* Minor fixes to the decode.py
2023-06-03 21:28:49 +08:00
Yifan Yang
ca60ced213
Fix typo (#1114)
* Fix typo for zipformer

* Fix typo for pruned_transducer_stateless7

* Fix typo for pruned_transducer_stateless7_ctc

* Fix typo for pruned_transducer_stateless7_ctc_bs

* Fix typo for pruned_transducer_stateless7_streaming

* Fix typo for pruned_transducer_stateless7_streaming_multi

* Fix file permissions for pruned_transducer_stateless7_streaming_multi

* Fix typo for pruned_transducer_stateless8

* Fix typo for pruned_transducer_stateless6

* Fix typo for pruned_transducer_stateless5

* Fix typo for pruned_transducer_stateless4

* Fix typo for pruned_transducer_stateless3
2023-06-02 14:12:42 +08:00
Yifan Yang
82f34a2388
Remove multidataset from librispeech/pruned_transducer_stateless7 (#1105)
* Add People's Speech to multidataset

* update

* remove multi from librispeech
2023-06-01 18:45:20 +08:00
Zengwei Yao
7a604057f9
update diagnostics, print limits in Balancer, merge changes from Dan's branch zlm59 (#1109) 2023-06-01 14:24:19 +08:00
Yifan Yang
03853f1ee5
Add peoples_speech (#1101)
* update

* Small fix

* Update egs/peoples_speech/ASR/prepare.sh

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>

* limit normalize log

* Update egs/peoples_speech/ASR/local/compute_fbank_peoples_speech_valid_test.py

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>

* Update compute_fbank_peoples_speech_splits.py

* Update compute_fbank_peoples_speech_valid_test.py

---------

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-05-31 12:46:17 +08:00
Fangjun Kuang
7b0afbdc16
Remove cur_batch_idx (#1102) 2023-05-30 14:49:54 +08:00
Fangjun Kuang
1aeffa73bc
remove outdated code in train.py (#1096) 2023-05-25 07:47:38 +08:00
Peter Ross
af8907e1ec
Update pre-commit isort package to v5.11.5 (#1095) 2023-05-24 19:57:37 +08:00
Zengwei Yao
6826b076d4
add flops profiler, support for Zipformer encoder and Conformer encoder (#1093)
* add flops profiler, support for Zipformer encoder and Conformer encoder

* support for reworked conformer and old zipformer

* skip black check
2023-05-24 19:10:45 +08:00
Fangjun Kuang
1df71a6b38
add onnx export for stateless2 (#1086) 2023-05-23 16:11:00 +08:00
Fangjun Kuang
ea8b15309f
Add onnx export scripts for wenetspeech recipe. (#1085) 2023-05-23 13:32:14 +08:00
Fangjun Kuang
dbcf0b41db
Fix stateless7 training error (#1082) 2023-05-23 12:52:02 +08:00
marcoyang1998
585e7b224f
Aishell pruned_transducer_stateless7 (#962)
* Add pruned_transducer_stateless7 for Aishell

* update README.md

* update comments and small fixes
2023-05-23 11:04:33 +08:00
Yifan Yang
7c4ff66a3d
Fix yesno Cl test (#1078) 2023-05-22 12:46:43 +08:00
Yifan Yang
90c392b7b3
Add docs for Fine-tune with mux (#1074)
* Update RESULTS.md
2023-05-22 12:39:51 +08:00
Fangjun Kuang
3883e362ad
Fix yesno CI test (#1077) 2023-05-22 12:29:51 +08:00
Zengwei Yao
8070258ec5
fix conv_emformer2, when using right_context_length=0 (#1076) 2023-05-21 20:31:54 +08:00
Zengwei Yao
30fcd16c7d
rm zipformer/__init__.py (#1075) 2023-05-20 23:12:11 +08:00
Zengwei Yao
a7e142b7ff
Support long audios recognition (#980)
* support long file transcription

* rename recipe as long_file_recog

* add docs

* support multi-gpu decoding

* style fix
2023-05-19 20:27:55 +08:00
Zengwei Yao
f18b539fbc
Add the upgraded Zipformer model (#1058)
* add the zipformer codes, copied from branch from_dan_scaled_adam_exp1119

* support model export with torch.jit.script

* update RESULTS.md

* support exporting streaming model with torch.jit.script

* add results of streaming models, with some minor changes

* update README.md

* add CI test

* update k2 version in requirements-ci.txt

* update pyproject.toml
2023-05-19 16:47:59 +08:00
Fangjun Kuang
a5bbfc6f7e
Update doc for exporting to ncnn (#1072) 2023-05-19 16:22:08 +08:00
Fangjun Kuang
ae1949ddcc
Support using the latest master from tencent/ncnn (#1070)
* Support using the latest master from tencent/ncnn

* small fixes
2023-05-18 20:56:58 +08:00
Yifan Yang
562bda91e4
Add adaption recipe for pruned_transducer_stateless7 (#1059)
* Add mux for finetune

* Add comments

* Fix for black

* Update finetune.py
2023-05-17 16:02:27 +08:00
Wei Kang
bccd20d978
Traning with byte level BPE (TAL_CSASR) (#1033)
* Add byte level bpe tal_csasr recipe

* Minor fixes to decoding and exporting

* Fix prepare.sh

* Update results
2023-05-16 12:44:52 +08:00
tomato18463
7a9f40aac5
Update the yesno recipe logs in doc (#1060) 2023-05-15 11:16:53 +08:00
arbs-gpu
30bde4b788
fix rnn_lm/train.py usage (#1055) 2023-05-11 17:37:47 +08:00