mirror of
https://github.com/k2-fsa/icefall.git
synced 2025-08-09 10:02:22 +00:00
53f38c01d2
* copy files from existing branch * add rule in .flake8 * monir style fix * fix typos * add tail padding * refactor, use fixed-length cache for batch decoding * copy from streaming branch * copy from streaming branch * modify emformer states stack and unstack, streaming decoding, to be continued * refactor Stream class * remane streaming_feature_extractor.py * refactor streaming decoding * test states stack and unstack * fix bugs, no grad, and num_proccessed_frames * add modify_beam_search, fast_beam_search * support torch.jit.export * use torch.div * copy from pruned_transducer_stateless4 * modify export.py * add author info * delete other test functions * minor fix * modify doc * fix style * minor fix doc * minor fix * minor fix doc * update RESULTS.md * fix typo * add info * fix typo * fix doc * add test function for conv module, and minor fix. * add copyright info * minor change of test_emformer.py * fix doc of stack and unstack, test case with batch_size=1 * update README.md
Introduction
Please refer to https://icefall.readthedocs.io/en/latest/recipes/librispeech/index.html for how to run models in this recipe.
./RESULTS.md contains the latest results.
Transducers
There are various folders containing the name transducer in this folder.
The following table lists the differences among them.
| Encoder | Decoder | Comment | |
|---|---|---|---|
transducer |
Conformer | LSTM | |
transducer_stateless |
Conformer | Embedding + Conv1d | Using optimized_transducer from computing RNN-T loss |
transducer_stateless2 |
Conformer | Embedding + Conv1d | Using torchaudio for computing RNN-T loss |
transducer_lstm |
LSTM | LSTM | |
transducer_stateless_multi_datasets |
Conformer | Embedding + Conv1d | Using data from GigaSpeech as extra training data |
pruned_transducer_stateless |
Conformer | Embedding + Conv1d | Using k2 pruned RNN-T loss |
pruned_transducer_stateless2 |
Conformer(modified) | Embedding + Conv1d | Using k2 pruned RNN-T loss |
pruned_transducer_stateless3 |
Conformer(modified) | Embedding + Conv1d | Using k2 pruned RNN-T loss + using GigaSpeech as extra training data |
pruned_transducer_stateless4 |
Conformer(modified) | Embedding + Conv1d | same as pruned_transducer_stateless2 + save averaged models periodically during training |
pruned_transducer_stateless5 |
Conformer(modified) | Embedding + Conv1d | same as pruned_transducer_stateless4 + more layers + random combiner |
pruned_transducer_stateless6 |
Conformer(modified) | Embedding + Conv1d | same as pruned_transducer_stateless4 + distillation with hubert |
pruned_stateless_emformer_rnnt2 |
Emformer(from torchaudio) | Embedding + Conv1d | Using Emformer from torchaudio for streaming ASR |
conv_emformer_transducer_stateless |
Emformer | Embedding + Conv1d | Using Emformer augmented with convolution for streaming ASR + mechanisms in reworked model |
The decoder in transducer_stateless is modified from the paper
Rnn-Transducer with Stateless Prediction Network.
We place an additional Conv1d layer right after the input embedding layer.