5fe58de43c
* initial commit * support download, data prep, and fbank * on-the-fly feature extraction by default * support BPE based lang * support HLG for BPE * small fix * small fix * chunked feature extraction by default * Compute features for GigaSpeech by splitting the manifest. * Fixes after review. * Split manifests into 2000 pieces. * set audio duration mismatch tolerance to 0.01 * small fix * add conformer training recipe * Add conformer.py without pre-commit checking * lazy loading and use SingleCutSampler * DynamicBucketingSampler * use KaldifeatFbank to compute fbank for musan * use pretrained language model and lexicon * use 3gram to decode, 4gram to rescore * Add decode.py * Update .flake8 * Delete compute_fbank_gigaspeech.py * Use BucketingSampler for valid and test dataloader * Update params in train.py * Use bpe_500 * update params in decode.py * Decrease num_paths while CUDA OOM * Added README * Update RESULTS * black * Decrease num_paths while CUDA OOM * Decode with post-processing * Update results * Remove lazy_load option * Use default `storage_type` * Keep the original tolerance * Use split-lazy * black * Update pretrained model Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
Installation
Please refer to https://icefall.readthedocs.io/en/latest/installation/index.html for installation.
Recipes
Please refer to https://icefall.readthedocs.io/en/latest/recipes/index.html for more information.
We provide four recipes at present:
yesno
This is the simplest ASR recipe in icefall and can be run on CPU.
Training takes less than 30 seconds and gives you the following WER:
[test_set] %WER 0.42% [1 / 240, 0 ins, 1 del, 0 sub ]
We do provide a Colab notebook for this recipe.
LibriSpeech
We provide 4 models for this recipe:
- conformer CTC model
- TDNN LSTM CTC model
- Transducer: Conformer encoder + LSTM decoder
- Transducer: Conformer encoder + Embedding decoder
Conformer CTC Model
The best WER we currently have is:
| test-clean | test-other | |
|---|---|---|
| WER | 2.42 | 5.73 |
We provide a Colab notebook to run a pre-trained conformer CTC model:
TDNN LSTM CTC Model
The WER for this model is:
| test-clean | test-other | |
|---|---|---|
| WER | 6.59 | 17.69 |
We provide a Colab notebook to run a pre-trained TDNN LSTM CTC model:
Transducer: Conformer encoder + LSTM decoder
Using Conformer as encoder and LSTM as decoder.
The best WER with greedy search is:
| test-clean | test-other | |
|---|---|---|
| WER | 3.07 | 7.51 |
We provide a Colab notebook to run a pre-trained RNN-T conformer model:
Transducer: Conformer encoder + Embedding decoder
Using Conformer as encoder. The decoder consists of 1 embedding layer and 1 convolutional layer.
The best WER using modified beam search with beam size 4 is:
| test-clean | test-other | |
|---|---|---|
| WER | 2.56 | 6.27 |
Note: No auxiliary losses are used in the training and no LMs are used in the decoding.
We provide a Colab notebook to run a pre-trained transducer conformer + stateless decoder model:
Aishell
We provide two models for this recipe: conformer CTC model and TDNN LSTM CTC model.
Conformer CTC Model
The best CER we currently have is:
| test | |
|---|---|
| CER | 4.26 |
We provide a Colab notebook to run a pre-trained conformer CTC model:
Transducer Stateless Model
The best CER we currently have is:
| test | |
|---|---|
| CER | 4.68 |
We provide a Colab notebook to run a pre-trained TransducerStateless model:
TDNN LSTM CTC Model
The CER for this model is:
| test | |
|---|---|
| CER | 10.16 |
We provide a Colab notebook to run a pre-trained TDNN LSTM CTC model:
TIMIT
We provide two models for this recipe: TDNN LSTM CTC model and TDNN LiGRU CTC model.
TDNN LSTM CTC Model
The best PER we currently have is:
| TEST | |
|---|---|
| PER | 19.71% |
We provide a Colab notebook to run a pre-trained TDNN LSTM CTC model:
TDNN LiGRU CTC Model
The PER for this model is:
| TEST | |
|---|---|
| PER | 17.66% |
We provide a Colab notebook to run a pre-trained TDNN LiGRU CTC model:
TED-LIUM3
We provide two models for this recipe: Transducer Stateless: Conformer encoder + Embedding decoder and Pruned Transducer Stateless: Conformer encoder + Embedding decoder + k2 pruned RNN-T loss.
Transducer Stateless: Conformer encoder + Embedding decoder
The best WER using modified beam search with beam size 4 is:
| dev | test | |
|---|---|---|
| WER | 6.91 | 6.33 |
Note: No auxiliary losses are used in the training and no LMs are used in the decoding.
We provide a Colab notebook to run a pre-trained Transducer Stateless model:
Pruned Transducer Stateless: Conformer encoder + Embedding decoder + k2 pruned RNN-T loss
The best WER using modified beam search with beam size 4 is:
| dev | test | |
|---|---|---|
| WER | 6.77 | 6.14 |
We provide a Colab notebook to run a pre-trained Pruned Transducer Stateless model:
Deployment with C++
Once you have trained a model in icefall, you may want to deploy it with C++, without Python dependencies.
Please refer to the documentation https://icefall.readthedocs.io/en/latest/recipes/librispeech/conformer_ctc.html#deployment-with-c for how to do this.
We also provide a Colab notebook, showing you how to run a torch scripted model in k2 with C++.
Please see: