Fangjun Kuang
34e40a86b3
Fix exporting decoder model to onnx ( #1264 )
...
* Use torch.jit.script() to export the decoder model
See also https://github.com/k2-fsa/sherpa-onnx/issues/327
2023-09-22 09:57:15 +08:00
Fangjun Kuang
f5dc957d44
Fix CI tests ( #1266 )
2023-09-21 21:16:14 +08:00
zr_jin
bbb03f7962
Update decoder.py ( #1262 )
2023-09-20 08:15:54 +08:00
zr_jin
7cc2dae940
Fixes to incorporate with the latest Lhotse release ( #1249 )
2023-09-13 12:39:49 +08:00
zr_jin
d50a9ea030
doc str fixes ( #1241 )
2023-09-07 16:34:53 +08:00
zr_jin
a81396b482
Use tokens.txt to replace bpe.model ( #1162 )
2023-08-12 16:53:59 +08:00
Fangjun Kuang
1dbbd7759e
Add tests for subsample.py and fix typos ( #1180 )
2023-07-25 14:46:18 +08:00
Peter Ross
b4c38d7547
Use symlinks for best epochs ( #1123 )
...
* utils: add symlink_or_copyfile
* pruned_transducer_stateless7: use symlinks (when possible) to output best epochs
* Rename function
---------
Co-authored-by: Yifan Yang <64255737+yfyeung@users.noreply.github.com>
2023-06-12 13:51:46 +08:00
Yifan Yang
ca60ced213
Fix typo ( #1114 )
...
* Fix typo for zipformer
* Fix typo for pruned_transducer_stateless7
* Fix typo for pruned_transducer_stateless7_ctc
* Fix typo for pruned_transducer_stateless7_ctc_bs
* Fix typo for pruned_transducer_stateless7_streaming
* Fix typo for pruned_transducer_stateless7_streaming_multi
* Fix file permissions for pruned_transducer_stateless7_streaming_multi
* Fix typo for pruned_transducer_stateless8
* Fix typo for pruned_transducer_stateless6
* Fix typo for pruned_transducer_stateless5
* Fix typo for pruned_transducer_stateless4
* Fix typo for pruned_transducer_stateless3
2023-06-02 14:12:42 +08:00
Yifan Yang
82f34a2388
Remove multidataset from librispeech/pruned_transducer_stateless7 ( #1105 )
...
* Add People's Speech to multidataset
* update
* remove multi from librispeech
2023-06-01 18:45:20 +08:00
Fangjun Kuang
7b0afbdc16
Remove cur_batch_idx ( #1102 )
2023-05-30 14:49:54 +08:00
Zengwei Yao
6826b076d4
add flops profiler, support for Zipformer encoder and Conformer encoder ( #1093 )
...
* add flops profiler, support for Zipformer encoder and Conformer encoder
* support for reworked conformer and old zipformer
* skip black check
2023-05-24 19:10:45 +08:00
Fangjun Kuang
dbcf0b41db
Fix stateless7 training error ( #1082 )
2023-05-23 12:52:02 +08:00
Yifan Yang
562bda91e4
Add adaption recipe for pruned_transducer_stateless7 ( #1059 )
...
* Add mux for finetune
* Add comments
* Fix for black
* Update finetune.py
2023-05-17 16:02:27 +08:00
Fangjun Kuang
5b50ffda54
support using mini librispeech in training ( #1048 )
...
* support mini librispeech in training
* update onnx export doc
2023-05-09 15:10:06 +08:00
Yifan Yang
2096e69bda
Use CutSet.mux for multidataset ( #1020 )
...
* Use CutSet.mux
* Remove mischange
* Fix for style check
2023-04-23 18:41:44 +08:00
Yifan Yang
d67a49afe4
Add multidataset ( #1010 )
...
* Add Common Voice for multidataset
* Add prepare_multidataset.sh
* Add dataset mixing
* Update prepare_multidataset.sh
* Update prepare_giga_speech.sh
* update comments
* Add split and shuffle mechanism
* Add multi-dataset train
* Fix for deleting
* Fix for modifying
* Add comments
* Change type for perturb_speed
* Fix for style check
* Small fix
* Add filter
* Remove warning
2023-04-21 18:09:41 +08:00
marcoyang1998
57d6482a79
Streaming Zipformer with multi-dataset ( #984 )
...
* modify train.py
* add right padding option in decode.py
* update RESULTS.md
2023-04-21 15:43:28 +08:00
Zengwei Yao
5f066d3d53
support decoding and computing RTF on test sets with onnx models ( #995 )
...
* support decode and compute RTF on test sets with onnx models
* support onnx export and decode in pruned_transducer_stateless
2023-04-12 19:04:50 +08:00
marcoyang1998
d337398d29
Shallow fusion for Aishell ( #954 )
...
* add shallow fusion and LODR for aishell
* update RESULTS
* add save by iterations
2023-04-03 16:20:29 +08:00
Yifan Yang
46bf6df62f
Remove simulate streaming from stateless7 ( #983 )
...
* Remove simulate streaming from stateless7
2023-04-03 14:55:45 +08:00
Fangjun Kuang
a632b24c35
Export int8 quantized models for non-streaming Zipformer. ( #977 )
...
* Export int8 quantized models for non-streaming Zipformer.
* Delete export-onnx.py
* Export int8 models for other folders
2023-03-31 22:46:19 +08:00
Zengwei Yao
bcc5923ab9
Support batch-wise forced-alignment ( #970 )
...
* support batch-wise forced-alignment based on beam search
* add length_norm to HypothesisList.topk()
* Use Hypothesis and HypothesisList instead
2023-03-28 23:24:24 +08:00
Fangjun Kuang
8c3ea93fc8
Save meta data to exported ONNX models ( #968 )
2023-03-27 11:39:29 +08:00
Zengwei Yao
7155769c19
minor fix, remove numel = p.numel() in optim.py ( #967 )
2023-03-24 15:30:29 +08:00
Peng He
f260a09ed4
remove if-branch at downsample pad in zipformer for onnx-export compatibility ( #965 )
2023-03-24 14:30:43 +08:00
marcoyang1998
7948624a22
Support fine-tuning ( #944 )
...
* support finetune
* add files for decoding giga
* support initializing modules
* add a fine-tune bash script
2023-03-17 13:44:29 +08:00
marcoyang1998
9ddd811925
Fix padding_idx ( #942 )
...
* fix padding_idx
* update RESULTS.md
2023-03-10 14:37:28 +08:00
Fangjun Kuang
f5de2e90c6
Fix style issues. ( #937 )
2023-03-08 22:56:04 +08:00
pehonnet
07243d136a
remove key from result filename ( #936 )
...
Co-authored-by: pe-honnet <pe.honnet@telepathy.ai>
2023-03-08 21:06:07 +08:00
Fangjun Kuang
c5e687ddf5
Export streaming zipformer to ncnn ( #906 )
2023-02-13 23:41:43 +08:00
Yifan Yang
c34ee67691
Update generate_model_from_checkpoint.py ( #901 )
2023-02-13 14:05:38 +08:00
Fangjun Kuang
2b995639b7
Add ONNX support for Zipformer and ConvEmformer ( #884 )
2023-02-09 00:02:38 +08:00
Yifan Yang
ffbf6d9199
Add generate_averaged_model.py ( #882 )
2023-02-07 16:19:08 +08:00
Zengwei Yao
1e6d6f8160
shuffle full Librispeech for zipformer recipes ( #869 )
...
* shuffle libri
2023-02-03 11:54:57 +08:00
Teo Wen Shen
1ce2bc1ee0
edit comments ( #852 )
2023-01-28 13:47:21 +08:00
Zengwei Yao
6b1ab71dc9
hardcode --filter-uneven-sized-batch ( #854 )
2023-01-27 21:24:12 +08:00
Zengwei Yao
2a463a420d
Filter uneven-sized batch ( #843 )
...
* add filter_uneven_sized_batch fucntion
* set --filter-uneven-sized-batch=True as default
2023-01-16 20:15:35 +08:00
Yunusemre
0f26edfde9
Add Zipformer Onnx Support ( #778 )
...
* add export script
* add zipformer onnx pretrained script
* add onnx zipformer test
* fix style
* add zipformer onnx to workflow
* replace is_in_onnx_export with is_tracing
* add github.event.label.name == 'onnx'
* add is_tracing to necessary conditions
* fix pooling_mask
* add onnx_check
* add onnx_check to scripts
* add is_tracing to scaling.py
2023-01-03 16:59:44 +08:00
Zengwei Yao
d167aad4ab
Add streaming zipformer ( #787 )
...
* add streaming zipformer codes
* add test_model.py
* add export.py, pretrained.py, jit_pretrained.py
* add cached_len for pooling module
* add jit_trace_export.py and jit_trace_pretrained.py
* fix bug in jit.trace
* update RESULTS.md
* add CI test
* minor fix in pruned_transducer_stateless7/zipformer.py
* update README.md
2022-12-30 10:52:18 +08:00
marcoyang1998
1f0408b103
Support Transformer LM ( #750 )
...
* support transformer LM
* show number of parameters during training
* update docstring
* testing files for ppl calculation
* add lm wrampper for rnn and transformer LM
* apply lm wrapper in lm shallow fusion
* small updates
* update decode.py to support LM fusion and LODR
* add export.py
* update CI and workflow
* update decoding results
* fix CI
* remove transformer LM from CI test
2022-12-29 10:53:36 +08:00
Yifan Yang
070c77e724
Add Blankskip to Zipformer+CTC ( #730 )
...
* init files
* add ctc as auxiliary loss and ctc_decode.py
* tuning the scalar of HLG score for 1best, nbest and nbest-oracle
* rename to pruned_transducer_stateless7_ctc
* fix doc
* fix bug, recover the hlg scores
* modify ctc_decode.py, move out the hlg scale
* fix hlg_scale
* add export.py and pretrained.py, and so on
* upload files, update README.md and RESULTS.md
* add CI test
* update .gitignore
* create symlinks
* Add Blank Skip to Zipformer+CTC
* Add warmup to blank skip
* Add warmup to blank skip
* Add __init__.py
* Add parameters_names to Adam
* Add warmup to blank skip
* Modify frame_reducer
* Modify frame_reducer
* Add Blank Skip to decode.
* Add ctc_decode.py
* Add blank skip to Zipformer+CTC
* process conflict
* process conflict
* modify ctc_guild_decode_bk.py
* modify Lconv
* produce the conflict
* Add export.py
* finish export
* fix for running black
* Add ci test
* Add ci-test
* chmod
* chmod
* fix bug for ci-test
* fix bug for ci-test
* fix bug for ci-test
* rename the dirname
* rename the dirname
* change dirname
* change dirname
* fix notes
* add pretrained.py
* add pretrained.py
* add pretrained.py
* add pretrained.py
* add pretrained.py
* add pretrained.py
* fix
* fix
* fix
* finished
* add the Copyright info and notes
Co-authored-by: Zengwei Yao <yaozengwei@outlook.com>
Co-authored-by: yifanyang <yifanyeung@yifanyangs-MacBook-Pro.local>
2022-12-21 17:41:31 +08:00
Zengwei Yao
65d7192dca
Fix zipformer attn_output_weights ( #774 )
...
* fix attn_output_weights
* remove in-place op
2022-12-19 20:10:39 +08:00
Zengwei Yao
fbc1d3b194
fix src_key_padding_mask in DownsampledZipformerEncoder ( #768 )
2022-12-17 22:03:13 +08:00
Zengwei Yao
0470bbae66
minor fix for zipformer recipe ( #758 )
...
* minor fix
* add CI test
2022-12-13 15:47:30 +08:00
Yifan Yang
02c18ba4b2
rm the dup line of Zipformer.py ( #755 )
...
Co-authored-by: yifanyang <yifanyeung@yifanyangs-MacBook-Pro.local>
2022-12-10 19:34:19 +08:00
Yifan Yang
a0cf85343d
fix for memory usage in pruned_transducer_stateless7/scaling.py ( #752 )
...
Co-authored-by: yifanyang <yifanyeung@yifanyangs-MacBook-Pro.local>
2022-12-09 19:23:11 +08:00
Guo Liyong
4fee3e7f1e
impove comment
2022-11-28 17:33:52 +08:00
Guo Liyong
9cf79cac3f
message formatting
2022-11-26 22:39:03 +08:00
Guo Liyong
89c3982a07
show dominant parameters
2022-11-26 00:50:21 +08:00
