639 Commits

Author SHA1 Message Date
jinzr
737274c9ed fixed few issues related to post processing 2023-08-12 01:04:27 +08:00
jinzr
bf6fb9f0e2 minor fixes 2023-08-11 21:05:36 +08:00
jinzr
14f0cb5977 minor bug fixes for existing scripts 2023-08-11 20:51:36 +08:00
zr_jin
2f4a0fd9fd fixed a formatting issue 2023-07-25 09:21:41 +08:00
jinzr
7e35a3b906 removed batch_name to fix a KeyError with "uttid" (#1172) 2023-07-24 23:55:16 +08:00
jinzr
0816be86ae updates for the zipformer_mmi and transducer_stateless recipes 2023-07-24 23:54:35 +08:00
jinzr
e0e8db3c91 updates for the pruned_transducer_stateless recipes 2023-07-24 23:54:35 +08:00
zr_jin
c03c011230 applied PR #1152 to other recipes 2023-07-24 23:54:35 +08:00
zr_jin
7e74c2d38b fixed a formatting issue 2023-07-24 23:54:35 +08:00
jinzr
8dcb6da8c7 updated the pruned_stateless_emformer_rnnt2 recipe 2023-07-24 23:54:35 +08:00
jinzr
d6f4805226 updated the lstm_transducer_stateless recipes
also revoked previous changes in conformer_ctc3/jit_pretrained.py
2023-07-24 23:54:35 +08:00
jinzr
64393e798f updated the conv_emformer_transducer_stateless recipes 2023-07-24 23:54:35 +08:00
jinzr
13bcfda1e4 updated all conformer_ctc* recipes to use tokens.txt in export.py and pretrained.py 2023-07-24 23:54:35 +08:00
jinzr
54c023034e Update pretrained.py 2023-07-24 23:54:35 +08:00
jinzr
9e79cf9f68 update the conformer_ctc recipe to replace lang-dir with tokens 2023-07-24 23:54:35 +08:00
jinzr
2edc3081d6 removed unused git lfs commands from librispeech zipformer recipe 2023-07-24 23:54:35 +08:00
zr_jin
5c0dfa52d2 Update egs/librispeech/ASR/pruned_transducer_stateless7/export-onnx.py
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-07-24 23:54:35 +08:00
jinzr
d3f6a8a392 Update onnx_pretrained-streaming.py 2023-07-24 23:54:35 +08:00
jinzr
c37ca4dd66 added tests for zipformer streaming & non-streaming export
details are listed below:
1. updated `git lfs` command in `export-onnx.py` and `onnx_pretrained-streaming.py`;
2. added rounding code to `export.py` for case where `params.avg == 1`.
2023-07-24 23:54:35 +08:00
jinzr
83e26a63e3 applied isort and removed unused imports 2023-07-24 23:54:35 +08:00
jinzr
ccb6031853 moved num_tokens to utils.py
moved `num_tokens` to `icefall/utils.py` to reduce code redundancy
2023-07-24 23:54:35 +08:00
jinzr
06cb1346ac fixed file permission 2023-07-24 23:54:35 +08:00
jinzr
f1fe409dee Update export-onnx.py
updated `export-onnx.py` to apply the same change as in [#1152](https://github.com/k2-fsa/icefall/pull/1152)
2023-07-24 23:54:35 +08:00
jinzr
aa2fc799c6 Update export-onnx.py
updated `export-oonx.py` to accept `tokens.txt` for blank_id and vocab_size
2023-07-24 23:54:35 +08:00
jinzr
cca02ae861 update for the pruned_transducer_stateless7 for aishell and librispeech 2023-07-24 23:54:35 +08:00
jinzr
fe5ffca1c1 init commit
init commit for an unified version of `export.py` and `pretrained.py`
2023-07-24 23:54:35 +08:00
Yifan Yang
ffe816e2a8
Fix blank skip ci test (#1167)
* Fix for ci

* Fix frame_reducer
2023-07-06 23:12:41 +08:00
Fangjun Kuang
130ad0319d
Fix CI test for zipformer CTC (#1165) 2023-07-05 10:38:29 +08:00
Fangjun Kuang
b8a17944e4
Fix zipformer CI test (#1164) 2023-07-05 10:23:35 +08:00
Fangjun Kuang
9009d028a0
Fix ONNX export for the latest non-streaming zipformer. (#1160) 2023-07-03 23:56:51 +08:00
Fangjun Kuang
c3e23ec8d2
Fix logaddexp for ONNX export (#1158) 2023-07-02 10:30:09 +08:00
MicKot
98d89463f6
zipformer2 logaddexp onnx safe (#1157) 2023-06-30 21:16:40 +08:00
Zengwei Yao
ccd8c624dd
support testing onnx exported model on the test sets (#1150)
* support testing onnx exported model on the test sets

* use token_table instead
2023-06-30 12:05:37 +08:00
Wei Kang
db71b03026
Support int8 quantization in decoder (#1152) 2023-06-29 16:48:59 +08:00
Desh Raj
9c2172c1c4
Zipformer for TedLium (#1125)
* initial commit for zipformer tedlium

* fix unk decoding

* add pretrained model and logs

* update for new AsrModel

* add option for choosing rnnt type

* add results with modified rnnt
2023-06-28 16:43:49 +08:00
Fangjun Kuang
968ebd236b
Fix ONNX export of the latest streaming zipformer model. (#1148) 2023-06-27 14:35:59 +08:00
Wei Kang
219bba1310
zipformer wenetspeech (#1130)
* copy files

* update train.py

* small fixes

* Add decode.py

* Fix dataloader in decode.py

* add blank penalty

* Add blank-penalty to other decoding method

* Minor fixes

* add zipformer2 recipe

* Minor fixes

* Remove pruned7

* export and test models

* Replace bpe with tokens in export.py and pretrain.py

* Minor fixes

* Minor fixes

* Minor fixes

* Fix export

* Update results

* Fix zipformer-ctc

* Fix ci

* Fix ci

* Fix CI

* Fix CI

---------

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-26 09:33:18 +08:00
frankyoujian
4d5b8369ae
fix small typo (#1144) 2023-06-21 17:17:19 +08:00
Yifan Yang
d667dc365b
Fix for diagnostic (#1135)
* CTC loss return tensor

* Update model.py
2023-06-16 15:04:41 +08:00
Yifan Yang
0a465794a8
Fix Zipformer (#1132)
* Update model.py

* Update train.py

* Update decoder.py
2023-06-15 17:52:14 +08:00
Fangjun Kuang
947f0614c9
Fix running exported model on GPU. (#1131) 2023-06-15 12:25:15 +08:00
Zengwei Yao
0ad037d076
Add CTC loss option in zipformer recipe (#1111)
* add CTC loss option in zipformer recipe

* add ctc_decode.py

* support CTC model export, add jit_pretrained_ctc.py, pretrained_ctc.py

* update README.md and RESULTS.md

* add CI test
2023-06-14 14:27:29 +08:00
danfu
0cb71ad3bc
add updated zipformer onnx export (#1108)
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-12 14:02:23 +08:00
Peter Ross
b4c38d7547
Use symlinks for best epochs (#1123)
* utils: add symlink_or_copyfile

* pruned_transducer_stateless7: use symlinks (when possible) to output best epochs

* Rename function

---------

Co-authored-by: Yifan Yang <64255737+yfyeung@users.noreply.github.com>
2023-06-12 13:51:46 +08:00
Yifan Yang
dca21c2a17
Fix parameters_names in train.py (#1121) 2023-06-08 16:54:05 +08:00
Fangjun Kuang
c0de78d3c0
Add data preparation for the MuST-C speech translation corpus (#1107) 2023-06-05 15:49:41 +08:00
Wei Kang
ba257efbcd
Add Context biasing (#1038)
* Add context biasing for librispeech

* Add context biasing for wenetspeech

* fix bugs

* Implement Aho-Corasick context graph

* fix some bugs

* Fixes to forward_one_step; add draw to context graph

* add output arc; fix black

* Fix wenetspeech tokenizer

* Minor fixes to the decode.py
2023-06-03 21:28:49 +08:00
Yifan Yang
ca60ced213
Fix typo (#1114)
* Fix typo for zipformer

* Fix typo for pruned_transducer_stateless7

* Fix typo for pruned_transducer_stateless7_ctc

* Fix typo for pruned_transducer_stateless7_ctc_bs

* Fix typo for pruned_transducer_stateless7_streaming

* Fix typo for pruned_transducer_stateless7_streaming_multi

* Fix file permissions for pruned_transducer_stateless7_streaming_multi

* Fix typo for pruned_transducer_stateless8

* Fix typo for pruned_transducer_stateless6

* Fix typo for pruned_transducer_stateless5

* Fix typo for pruned_transducer_stateless4

* Fix typo for pruned_transducer_stateless3
2023-06-02 14:12:42 +08:00
Yifan Yang
82f34a2388
Remove multidataset from librispeech/pruned_transducer_stateless7 (#1105)
* Add People's Speech to multidataset

* update

* remove multi from librispeech
2023-06-01 18:45:20 +08:00
Fangjun Kuang
7b0afbdc16
Remove cur_batch_idx (#1102) 2023-05-30 14:49:54 +08:00