431 Commits

Author SHA1 Message Date
yaozengwei
522a45ce75 add using averaged model in export.py 2022-08-10 15:43:50 +08:00
yaozengwei
45c7894111 support jit trace 2022-08-09 19:29:21 +08:00
yaozengwei
03b056ca37 add scaling converter for ScaledLSTM 2022-08-05 19:38:30 +08:00
yaozengwei
9e4b5bdfdf Merge remote-tracking branch 'k2-fsa/master' into lstm 2022-08-05 14:36:04 +08:00
Yunusemre
7157f62af3
Merging onnx models (#518)
* add export function of onnx-all-in-one to export.py

* add onnx_check script for all-in-one onnx model

* minor fix

* remove unused arguments

* add onnx-all-in-one test

* fix style

* fix style

* fix requirements

* fix input/output names

* fix installing onnx_graphsurgeon

* fix instaliing onnx_graphsurgeon

* revert to previous requirements.txt

* fix minor
2022-08-04 23:03:41 +08:00
Zengwei Yao
a4dd273776
fix about tensorboard (#516)
* fix metricstracker

* fix style
2022-08-04 19:57:12 +08:00
Fangjun Kuang
6af5a82d8f
Convert ScaledEmbedding to nn.Embedding for inference. (#517)
* Convert ScaledEmbedding to nn.Embedding for inference.

* Fix CI style issues.
2022-08-03 15:34:55 +08:00
Fangjun Kuang
58a96e5b68
Support exporting to ONNX format (#501)
* WIP: Support exporting to ONNX format

* Minor fixes.

* Combine encoder/decoder/joiner into a single file.

* Revert merging three onnx models into a single one.

It's quite time consuming to extract a sub-graph from the combined
model. For instance, it takes more than one hour to extract
the encoder model.

* Update CI to test ONNX models.

* Decode with exported models.

* Fix typos.

* Add more doc.

* Remove ncnn as it is not fully tested yet.

* Fix as_strided for streaming conformer.
2022-08-03 10:30:28 +08:00
Wei Kang
2f75236c05
Support dynamic chunk streaming training in pruned_transcuder_stateless5 (#454)
* support dynamic chunk streaming training

* Add simulate streaming decoding

* Support streaming decoding

* fix causal

* Minor fixes

* fix streaming decode; add results
2022-07-29 16:40:06 +08:00
Fangjun Kuang
ec69967584
Set overwrite=True when extracting features in batches. (#487) 2022-07-29 11:17:19 +08:00
Fangjun Kuang
4612b03947
Fix using G before assignment in pruned_transducer_stateless/decode.py (#494) 2022-07-26 10:37:02 +08:00
yaozengwei
6871c96ffa add using proj_size 2022-07-25 19:04:08 +08:00
Wei Kang
b1d0956855
Add modified_beam_search for streaming decode (#489)
* Add modified_beam_search for pruned_transducer_stateless/streaming_decode.py

* refactor

* modified beam search for stateless3,4

* Fix comments

* Add real streamng ci
2022-07-25 16:53:23 +08:00
Zengwei Yao
8203d10be7
Add stats about duration and padding proportion (#485)
* add stats about duration and padding proportion

* add  for utt_duration

* add stats for other recipes

* add stats for other 2 recipes

* modify doc

* minor change
2022-07-25 16:40:43 +08:00
Quandwang
116d0cf26d
CTC attention model with reworked Conformer encoder and reworked Transformer decoder (#462)
* ctc attention model with reworked conformer encoder and reworked transformer decoder

* remove unnecessary func

* resolve flake8 conflicts

* fix typos and modify the expr of ScaledEmbedding

* use original beam size

* minor changes to the scripts

* add rnn lm decoding

* minor changes

* check whether q k v weight is None

* check whether q k v weight is None

* check whether q k v weight is None

* style correction

* update results

* update results

* upload the decoding results of rnn-lm to the RESULTS

* upload the decoding results of rnn-lm to the RESULTS

* Update egs/librispeech/ASR/RESULTS.md

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>

* Update egs/librispeech/ASR/RESULTS.md

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>

* Update egs/librispeech/ASR/RESULTS.md

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2022-07-22 15:31:25 +08:00
ezerhouni
608473b4eb
Add RNN-LM rescoring in fast beam search (#475) 2022-07-18 16:52:17 +08:00
yaozengwei
9bb0c7988f add random combiner for training deeper model 2022-07-18 11:50:35 +08:00
yaozengwei
8bd700cff2 fix scale value in scaling.py 2022-07-18 10:33:35 +08:00
yaozengwei
3cedbe3678 fix style 2022-07-17 21:40:29 +08:00
yaozengwei
fd261eca3a Merge remote-tracking branch 'k2-fsa/master' into lstm_new 2022-07-17 21:24:56 +08:00
yaozengwei
c71788ee57 remove add_model_arguments 2022-07-17 21:20:39 +08:00
yaozengwei
7c00f92abb fix style 2022-07-17 21:17:45 +08:00
yaozengwei
872d2390d2 link __init__.py 2022-07-17 20:39:46 +08:00
yaozengwei
ce2d817114 pruned2 -> pruned4 2022-07-17 20:36:20 +08:00
yaozengwei
125eac8dee flat weights after scaling 2022-07-17 20:35:29 +08:00
yaozengwei
539a9d75d4 simplified code in ScaledLSTM 2022-07-17 17:07:14 +08:00
yaozengwei
5c669b7716 modify streaming decoding files 2022-07-17 16:09:24 +08:00
yaozengwei
822cc78a9c copy streaming decoding files from pruned_transducer_stateless2 2022-07-17 15:47:43 +08:00
yaozengwei
4a0dea2aa2 modify decode.py pretrained.py test_model.py train.py 2022-07-17 15:38:53 +08:00
yaozengwei
b1be6ea475 copy from pruned_transducer_stateless2 2022-07-17 15:37:27 +08:00
yaozengwei
89bfb6b9c7 link scaling.py pruned_transducer_stateless2 2022-07-17 15:35:59 +08:00
yaozengwei
d16b9ec15f link from pruned_transducer_stateless2 2022-07-17 15:32:54 +08:00
yaozengwei
074bd7da71 hardcode bidirectional=False 2022-07-17 15:31:25 +08:00
yaozengwei
2d53f2ef8b add RNN and Conv2dSubsampling classes in lstm.py 2022-07-17 12:59:27 +08:00
yaozengwei
7c9fcfa5c9 add RNNEncoderLayer and RNNEncoder classes in lstm.py 2022-07-16 22:50:42 +08:00
yaozengwei
9165de5f57 add ScaledLSTM 2022-07-16 22:47:05 +08:00
ezerhouni
ffca1ae7fb
[WIP] Rnn-T LM nbest rescoring (#471) 2022-07-15 10:32:54 +08:00
LIyong.Guo
f8d28f0998
update multi_quantization installation (#469)
* update multi_quantization installation

* Update egs/librispeech/ASR/pruned_transducer_stateless6/train.py

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>

Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2022-07-13 21:16:45 +08:00
Zengwei Yao
bc2882ddcc
Simplified memory bank for Emformer (#440)
* init files

* use average value as memory vector for each chunk

* change tail padding length from right_context_length to chunk_length

* correct the files, ln -> cp

* fix bug in conv_emformer_transducer_stateless2/emformer.py

* fix doc in conv_emformer_transducer_stateless/emformer.py

* refactor init states for stream

* modify .flake8

* fix bug about memory mask when memory_size==0

* add @torch.jit.export for init_states function

* update RESULTS.md

* minor change

* update README.md

* modify doc

* replace torch.div() with <<

* fix bug, >> -> <<

* use i&i-1 to judge if it is a power of 2

* minor fix

* fix error in RESULTS.md
2022-07-12 19:19:58 +08:00
Zengwei Yao
ce26495238
Rand combine update result (#467)
* update RESULTS.md

* fix test code in pruned_transducer_stateless5/conformer.py

* minor fix

* delete doc

* fix style
2022-07-11 18:13:31 +08:00
Zengwei Yao
d80f29e662
Modification about random combine (#452)
* comment some lines, random combine from 1/3 layers, on linear layers in combiner

* delete commented lines

* minor change
2022-06-30 12:23:49 +08:00
Wei Kang
6e609c67a2
Using streaming conformer as transducer encoder (#380)
* support streaming in conformer

* Add more documents

* support streaming on pruned_transducer_stateless2; add delay penalty; fixes for decode states

* Minor fixes

* streaming for pruned_transducer_stateless4

* Fix conv cache error, support async streaming decoding

* Fix style

* Fix style

* Fix style

* Add torch.jit.export

* mask the initial cache

* Cutting off invalid frames of encoder_embed output

* fix relative positional encoding in streaming decoding for compution saving

* Minor fixes

* Minor fixes

* Minor fixes

* Minor fixes

* Minor fixes

* Fix jit export for torch 1.6

* Minor fixes for streaming decoding

* Minor fixes on decode stream

* move model parameters to train.py

* make states in forward streaming optional

* update pretrain to support streaming model

* update results.md

* update tensorboard and pre-models

* fix typo

* Fix tests

* remove unused arguments

* add streaming decoding ci

* Minor fix

* Minor fix

* disable right context by default
2022-06-28 00:18:54 +08:00
Jun Wang
d792bdc9bc
fix typo (#445) 2022-06-25 11:00:53 +08:00
Tiance Wang
c0ea334738
fix bug of concatenating list to tuple (#444) 2022-06-24 19:31:09 +08:00
ezerhouni
0475d75d15
[Ready to be merged] Add RNN-LM to Conformer-CTC decoding (#439) 2022-06-23 19:37:03 +08:00
Fangjun Kuang
dc89b61b80
Add fast_beam_search_nbest. (#420)
* Add fast_beam_search_nbest.

* Fix CI errors.

* Fix CI errors.

* More fixes.

* Small fixes.

* Support using log_add in LG decoding with fast_beam_search.

* Support LG decoding in pruned_transducer_stateless

* Support LG for pruned_transducer_stateless2.

* Support LG for fast beam search.

* Minor fixes.
2022-06-22 00:09:25 +08:00
Fangjun Kuang
7100c33820
Add pruned RNN-T for aishell. (#436)
* Add pruned RNN-T for aishell.

* support torch script.

* Update CI.

* Minor fixes.

* Add links to sherpa.
2022-06-21 21:17:22 +08:00
Zengwei Yao
d3daeaf5cd
Upload extracted codebook indexes (#429)
* save only vq-related info to manifest

* support to join manifest files

* support using extracted codebook indexes

* fix doc

* minor fix

* add enable-distillation argument option, fix monir typos

* fix style

* fix typo
2022-06-21 19:16:59 +08:00
Zengwei Yao
a42d96dfe0
Fix warmup (#435)
* fix warmup when scan_pessimistic_batches_for_oom

* delete comments
2022-06-20 13:40:01 +08:00
Fangjun Kuang
ab788980c9
Fix an error introduced by supporting torchscript for torch 1.6.0 (#434) 2022-06-18 08:57:20 +08:00