mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-08-08 09:32:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
909 Commits 5 Branches 3 Tags
Commit Graph

909 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yifan Yang
98569b2607
Update RESULTS.md (#1036) 
* Update RESULTS.md
 2023-05-06 17:51:55 +08:00
Wei Kang
80156dda09
Training with byte level BPE (AIShell) (#986) 
* copy files from zipformer librispeech

* Add byte bpe training for aishell

* compile LG graph

* Support LG decoding

* Minor fixes

* black

* Minor fixes

* export & fix pretrain.py

* fix black

* Update RESULTS.md

* Fix export.py
 2023-05-04 19:16:17 +08:00
PF Luo
61ec3a7a8f
fix export RNNLM onnx model typo (#1029) 2023-04-28 19:53:06 +08:00
Yuanhang Zhang
b0228c536e
Fix typo in librispeech OpenFST-based HLG preparation script (#1028) 2023-04-28 19:52:32 +08:00
PF Luo
298ed4520f
add meta-data embedding_dim to RNNLM onnx-model (#1026) 2023-04-28 16:33:46 +08:00
Fangjun Kuang
2767b9ff11
Support exporting RNNLM to ONNX. (#1014) 
* Support exporting RNNLM to ONNX.

* add int8 models

* fix style issues

* Fix EOS padding

* support exporting for streaming ASR
 2023-04-27 14:36:36 +08:00
marcoyang1998
45c13e90e4
RNNLM rescore + Low-order density ratio (#1017) 
* add rnnlm rescore + LODR

* add LODR in decode.py

* update RESULTS
 2023-04-24 15:00:02 +08:00
Yifan Yang
2096e69bda
Use CutSet.mux for multidataset (#1020) 
* Use CutSet.mux

* Remove mischange

* Fix for style check
 2023-04-23 18:41:44 +08:00
Yifan Yang
d67a49afe4
Add multidataset (#1010) 
* Add Common Voice for multidataset

* Add prepare_multidataset.sh

* Add dataset mixing


* Update prepare_multidataset.sh

* Update prepare_giga_speech.sh

* update comments

* Add split and shuffle mechanism

* Add multi-dataset train

* Fix for deleting

* Fix for modifying

* Add comments

* Change type for perturb_speed

* Fix for style check

* Small fix

* Add filter

* Remove warning
 2023-04-21 18:09:41 +08:00
marcoyang1998
57d6482a79
Streaming Zipformer with multi-dataset (#984) 
* modify train.py

* add right padding option in decode.py

* update RESULTS.md
 2023-04-21 15:43:28 +08:00
Wei Kang
0efed1cec5
Fix path in aishell rnnlm training (#1016) 2023-04-20 23:09:31 +08:00
Wei Kang
5c65516e05
Fix aishell rnnlm training command (#1015) 2023-04-20 16:14:16 +08:00
Yifan Yang
81d386ef3e
Add compute_ppl.py and ngram_entropy_pruning.py (#1013) 2023-04-20 12:27:43 +08:00
Wen Ding
78b9dcc936
Support exporting BS Zipformer models to ONNX, used in Triton Server (#1008) 
* Support export BS Zipformer models to ONNX in Tritron

* Update copyright

* Update exporting codes for BS zipformer models

* Code format

* Update comments

* Update export_onnx.py

---------

Co-authored-by: Yifan Yang <64255737+yfyeung@users.noreply.github.com>
 2023-04-18 17:05:08 +08:00
Yifan Yang
05e7435d0d
Move soft links into proper position (#1007) 2023-04-18 10:11:12 +08:00
Yifan Yang
8838fe0bd2
Zipformer for Common Voice (#997) 
* Add soft links in pruned_transducer_stateless7 for CommonVoice

* Add python files

* Update prepare.sh

* Update normalization

* Fix for soft links

* Add some docs

* Add export

* Update egs/commonvoice/ASR/RESULTS.md

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>

* Add export for onnx

---------

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
 2023-04-17 17:47:25 +08:00
marcoyang1998
34d1b07c3d
Modified beam search with RNNLM rescoring (#1002) 
* add RNNLM rescore

* add shallow fusion and lm rescore for streaming zipformer

* minor fix

* update RESULTS.md

* fix yesno workflow, change from ubuntu-18.04 to ubuntu-latest
 2023-04-17 16:43:00 +08:00
Fangjun Kuang
e32658e620
Fix torch.jit.script() export for streaming zipformer. (#1005) 2023-04-17 16:13:30 +08:00
Zengwei Yao
7c7d9ab042
add @torch.jit.export for streaming_forward func in Zipformer class (#1004) 2023-04-17 12:03:52 +08:00
Zengwei Yao
5f066d3d53
support decoding and computing RTF on test sets with onnx models (#995) 
* support decode and compute RTF on test sets with onnx models

* support onnx export and decode in pruned_transducer_stateless
 2023-04-12 19:04:50 +08:00
Yifan Yang
dbf2aa3212
Create preprocess_commonvoice.py (#996) 2023-04-11 21:04:54 +08:00
Yifan Yang
3cb0a0121b
Add Common Voice (#994) 
* Add commonvoice

* Add data preparation recipe

* Updata

* update prepare.sh

* Fix for black

* Update prefix with cv-

* 20 ->

* Update compute_fbank_commonvoice_dev_test.py

* Update prepare.sh

* Update compute_fbank_commonvoice_dev_test.py
 2023-04-11 20:56:40 +08:00
Yifan Yang
33578cca48
Fix filter_cuts in compute_fbank_librispeech.py (#993) 2023-04-11 11:12:05 +08:00
Yifan Yang
6434c8eadc
Add averaged model && change start from 0 to 1 && fix typo for gigaspeech (#990) 
* Add averaged model && change start from 0 to 1 && fix typo

* Update train.py

* Set use-averaged-model False for BC

---------

Co-authored-by: yifanyang <yifanyeung@yifanyangs-MacBook-Pro.local>
 2023-04-09 20:53:47 +08:00
Zengwei Yao
136aa94d57
remove duplicated lines (#988) 2023-04-06 17:47:33 +08:00
Yifan Yang
c90f57afdb
Remove simulate streaming from stateless8 (#985) 2023-04-04 11:04:00 +08:00
marcoyang1998
d337398d29
Shallow fusion for Aishell (#954) 
* add shallow fusion and LODR for aishell

* update RESULTS

* add save by iterations
 2023-04-03 16:20:29 +08:00
Yifan Yang
46bf6df62f
Remove simulate streaming from stateless7 (#983) 
* Remove simulate streaming from stateless7
 2023-04-03 14:55:45 +08:00
Yifan Yang
180c7c2b7a
Add UniqueLexicon for gigaspeech (#982) 2023-04-03 12:39:34 +08:00
Yifan Yang
12a222aa4b
Fix comments on the usage of train.py (#981) 2023-04-02 16:32:43 +08:00
Fangjun Kuang
a632b24c35
Export int8 quantized models for non-streaming Zipformer. (#977) 
* Export int8 quantized models for non-streaming Zipformer.

* Delete export-onnx.py

* Export int8 models for other folders
 2023-03-31 22:46:19 +08:00
marcoyang1998
c21b6a208b
Add finetuning script for aishell (#974) 
* add aishell finetune scripts

* add an example bash script
 2023-03-30 17:08:46 +08:00
Zengwei Yao
2a5a75cb56
add option of using full attention for streaming model decoding (#975) 2023-03-30 14:30:13 +08:00
Zengwei Yao
bcc5923ab9
Support batch-wise forced-alignment (#970) 
* support batch-wise forced-alignment based on beam search

* add length_norm to HypothesisList.topk()

* Use Hypothesis and HypothesisList instead
 2023-03-28 23:24:24 +08:00
PF Luo
15d48e3a6a
fix rnn_lm && transformer_lm import problem (#971) 2023-03-28 19:14:08 +08:00
Fangjun Kuang
35e21a0d2e
Fix torchscript export for aishell (#969) 2023-03-27 14:08:26 +08:00
Fangjun Kuang
8c3ea93fc8
Save meta data to exported ONNX models (#968) 2023-03-27 11:39:29 +08:00
Zengwei Yao
7155769c19
minor fix, remove numel = p.numel() in optim.py (#967) 2023-03-24 15:30:29 +08:00
Peng He
f260a09ed4
remove if-branch at downsample pad in zipformer for onnx-export compatibility (#965) 2023-03-24 14:30:43 +08:00
Wei Kang
d74822d07b
Fix wenetspeech decoding speed (#953) 2023-03-21 21:35:32 +08:00
marcoyang1998
7948624a22
Support fine-tuning (#944) 
* support finetune

* add files for decoding giga

* support initializing modules

* add a fine-tune bash script
 2023-03-17 13:44:29 +08:00
Jason's Lab
6196b4a407
Add char-based language model training process for aishell. (#945) 
* Add char-based language model training process for aishell.

Add soft link from librispeech/ASR/local/sort_lm_training_data.py to aishell/ASR/local/

---------

Co-authored-by: lichao <www.563042811@qq.com>
 2023-03-16 09:52:11 +08:00
Yifan Yang
a48812ddb3
Ban the test_rnn.py in ci-test (#949) 2023-03-15 22:02:20 +08:00
Yifan Yang
cad6735e07
Modify make_pad_mask to support TensorRT (#943) 
* Modify make_pad_mask to support TensorRT

* Fix for test
 2023-03-10 19:28:59 +08:00
marcoyang1998
9ddd811925
Fix padding_idx (#942) 
* fix padding_idx

* update RESULTS.md
 2023-03-10 14:37:28 +08:00
Yifan Yang
28af269e5e
Fix for workflow (#934) 2023-03-09 17:38:15 +08:00
Fangjun Kuang
f5de2e90c6
Fix style issues. (#937) 2023-03-08 22:56:04 +08:00
pehonnet
07243d136a
remove key from result filename (#936) 
Co-authored-by: pe-honnet <pe.honnet@telepathy.ai>
 2023-03-08 21:06:07 +08:00
Fangjun Kuang
8aaa9761e4
Add doc about exporting streaming zipformer to sherpa-ncnn (#927) 2023-02-27 21:23:04 +08:00
Fangjun Kuang
b7c85968ae
Use standard apache 2.0 license (#919) 2023-02-22 11:15:58 +08:00