456 Commits

Author SHA1 Message Date
pkufool
8d37175ffb make states in forward streaming optional 2022-06-25 17:44:16 +08:00
pkufool
61f3c87d48 Merge branch 'master' into streaming-conformer 2022-06-25 09:50:37 +08:00
pkufool
934f25f987 move model parameters to train.py 2022-06-25 09:39:39 +08:00
Tiance Wang
c0ea334738
fix bug of concatenating list to tuple (#444) 2022-06-24 19:31:09 +08:00
Mingshuang Luo
c391bfd100
fix errors for soft connection (#443) 2022-06-24 10:40:46 +08:00
ezerhouni
0475d75d15
[Ready to be merged] Add RNN-LM to Conformer-CTC decoding (#439) 2022-06-23 19:37:03 +08:00
Fangjun Kuang
dc89b61b80
Add fast_beam_search_nbest. (#420)
* Add fast_beam_search_nbest.

* Fix CI errors.

* Fix CI errors.

* More fixes.

* Small fixes.

* Support using log_add in LG decoding with fast_beam_search.

* Support LG decoding in pruned_transducer_stateless

* Support LG for pruned_transducer_stateless2.

* Support LG for fast beam search.

* Minor fixes.
2022-06-22 00:09:25 +08:00
Fangjun Kuang
7100c33820
Add pruned RNN-T for aishell. (#436)
* Add pruned RNN-T for aishell.

* support torch script.

* Update CI.

* Minor fixes.

* Add links to sherpa.
2022-06-21 21:17:22 +08:00
Zengwei Yao
d3daeaf5cd
Upload extracted codebook indexes (#429)
* save only vq-related info to manifest

* support to join manifest files

* support using extracted codebook indexes

* fix doc

* minor fix

* add enable-distillation argument option, fix monir typos

* fix style

* fix typo
2022-06-21 19:16:59 +08:00
pkufool
5fe60dec43 Minor fixes on decode stream 2022-06-20 20:01:47 +08:00
2xwwx2
91b2765cfd
Fixs spelling mistake (#438) 2022-06-20 16:41:04 +08:00
Mingshuang Luo
998091ef52
do some changes for export.py (#437) 2022-06-20 14:57:08 +08:00
Zengwei Yao
a42d96dfe0
Fix warmup (#435)
* fix warmup when scan_pessimistic_batches_for_oom

* delete comments
2022-06-20 13:40:01 +08:00
pkufool
5f64396112 Merge branch 'master' into streaming-conformer 2022-06-18 09:08:34 +08:00
Fangjun Kuang
ab788980c9
Fix an error introduced by supporting torchscript for torch 1.6.0 (#434) 2022-06-18 08:57:20 +08:00
pkufool
6a937ffa40 Minor fixes for streaming decoding 2022-06-18 08:41:52 +08:00
pkufool
88ed814197 Fix jit export for torch 1.6 2022-06-18 08:35:25 +08:00
Fangjun Kuang
d53f69108f
Support torch 1.6.0 (#433) 2022-06-17 22:24:47 +08:00
Wei Kang
5379c8e9fa
Disable drop_last in testing time (#427) 2022-06-16 15:43:48 +08:00
Mingshuang Luo
5c3ee8bfcd
[Ready to merge] Pruned transducer stateless5 recipe for AISHELL4 (#399)
* pruned-transducer-stateless5 recipe for aishell4

* pruned-transducer-stateless5 recipe for aishell4

* do some changes and text normalize

* do some changes

* add text normalize

* combine the training data and decode without webdataset

* update codes for merging

* Do a change for READMD.md
2022-06-14 22:19:05 +08:00
pkufool
413ca2ef77 Merge remote-tracking branch 'origin/master' into streaming-conformer 2022-06-13 19:10:44 +08:00
Zengwei Yao
53f38c01d2
Emformer with conv module and scaling mechanism (#389)
* copy files from existing branch

* add rule in .flake8

* monir style fix

* fix typos

* add tail padding

* refactor, use fixed-length cache for batch decoding

* copy from streaming branch

* copy from streaming branch

* modify emformer states stack and unstack, streaming decoding, to be continued

* refactor Stream class

* remane streaming_feature_extractor.py

* refactor streaming decoding

* test states stack and unstack

* fix bugs, no grad, and num_proccessed_frames

* add modify_beam_search, fast_beam_search

* support torch.jit.export

* use torch.div

* copy from pruned_transducer_stateless4

* modify export.py

* add author info

* delete other test functions

* minor fix

* modify doc

* fix style

* minor fix doc

* minor fix

* minor fix doc

* update RESULTS.md

* fix typo

* add info

* fix typo

* fix doc

* add test function for conv module, and minor fix.

* add copyright info

* minor change of test_emformer.py

* fix doc of stack and unstack, test case with batch_size=1

* update README.md
2022-06-13 15:09:17 +08:00
pkufool
85ddfd958c Minor fixes 2022-06-10 19:06:06 +08:00
Fangjun Kuang
9f6c748b30
Add links to sherpa. (#417)
* Add links to sherpa.
2022-06-10 12:19:18 +08:00
Fangjun Kuang
bfeab319c9
Fix aishell. (#416) 2022-06-10 11:47:43 +08:00
Fangjun Kuang
dbda1644b5
Replace load_manifest_lazy with load_manifest for MUSAN. (#412) 2022-06-09 11:42:18 +08:00
Fangjun Kuang
ed66877694
Replace ChunkedLilcomHdf5Writer with LilcomChunkyWriter. (#411) 2022-06-09 11:18:52 +08:00
Quandwang
8512aaf585
fix typos (#409) 2022-06-08 20:08:44 +08:00
Mingshuang Luo
5079d99ee2
a correction for text2segmentation.py (#407) 2022-06-08 12:06:57 +08:00
Fangjun Kuang
1094a3cb37
Replace LilcomChunkyWriter with ChunkedLilcomHdf5Writer. (#404) 2022-06-07 18:14:25 +08:00
pkufool
aebe9c22dd Minor fixes 2022-06-07 15:59:58 +08:00
pkufool
d7be9bd9c5 Minor fixes 2022-06-07 12:00:26 +08:00
Fangjun Kuang
80c46f0abd
Fix exporting emformer with torchscript using torch 1.6.0 (#402) 2022-06-07 09:19:37 +08:00
pkufool
1c794e32b0 Minor fixes 2022-06-06 20:19:49 +08:00
pkufool
09b0c54983 Minor fixes 2022-06-06 17:56:22 +08:00
Fangjun Kuang
29fa878fff
Fix Emformer for torchscript using torch 1.6.0 (#401) 2022-06-06 17:08:07 +08:00
Mingshuang Luo
0a21eaae7f
do a change for decode.py (#400) 2022-06-06 15:44:04 +08:00
Fangjun Kuang
f1abce72f8
Use jsonl for CutSet in the LibriSpeech recipe. (#397)
* Use jsonl for cutsets in the librispeech recipe.

* Use lazy cutset for all recipes.

* More fixes to use lazy CutSet.

* Remove force=True from logging to support Python < 3.8

* Minor fixes.

* Fix style issues.
2022-06-06 10:19:16 +08:00
pkufool
3aacf75652 fix relative positional encoding in streaming decoding for compution saving 2022-06-06 06:46:40 +08:00
Mingshuang Luo
e5884f82e0
[Ready to merge] Add prefix for compute fbank (#398)
* add prefix

* add prefix
2022-06-05 18:17:52 +08:00
fanlu
8a3068ead8
Update decode.py (#392)
* Update decode.py

fix bug ```TypeError: greedy_search_batch() missing 1 required positional argument: 'encoder_out_lens'```

* fix modified_beam_search

Co-authored-by: fanlu3 <fanlu@jd.com>
2022-06-04 19:08:17 +08:00
Zengwei Yao
148f69d8d9
Update RESULTS.md (#388)
* update RESULT.md about pruned_transducer_stateless4

* Update RESULT.md

This PR is only to update RESULT.md about pruned_transducer_stateless4.

* set default value of --use-averaged-model to True

* update RESULTS.md and add decode command

* minor fix

* update export.py

* add uploaded files links

* update link

* fix typos
2022-06-04 15:52:35 +08:00
Mingshuang Luo
beab229fd7
[Ready to merge] Pruned_transducer_stateless2 for alimeeting dataset (#378)
* add pruned-rnnt2 recipe for alimeeting dataset

* update code for merging

* change LilcomHdf5Writer to ChunkedLilcomHdf5Writer

* change for test.yml

* change for test.yml

* change for test.yml

* change for workflow yml

* change for yml

* change for yml

* change for README.md

* change for yml

* solve the conflicts

* solve the conflicts
2022-06-04 13:47:46 +08:00
pkufool
fc54a99a56 Cutting off invalid frames of encoder_embed output 2022-06-02 14:19:08 +08:00
pkufool
9629be124d mask the initial cache 2022-06-01 21:50:02 +08:00
Fangjun Kuang
fbfc98f1d3
Add streaming Emformer stateless RNN-T. (#390)
* Add streaming Emformer stateless RNN-T.

* Update results for streaming Emformer.

* Minor fixes.
2022-06-01 14:31:47 +08:00
pkufool
0325e3a04e Add torch.jit.export 2022-05-29 16:25:01 +08:00
pkufool
605838da55 Fix style 2022-05-29 07:48:52 +08:00
pkufool
aecaecfb17 Fix style 2022-05-29 07:35:52 +08:00
pkufool
fb3f3d2526 Fix style 2022-05-29 07:33:54 +08:00