From f5bf881196ee21c301edd17d42d4869dccabe4ad Mon Sep 17 00:00:00 2001 From: Fangjun Kuang Date: Tue, 24 Aug 2021 20:27:00 +0800 Subject: [PATCH] More doc. --- egs/librispeech/ASR/conformer_ctc/README.md | 353 +------------------- egs/yesno/ASR/README.md | 15 +- egs/yesno/ASR/tdnn/README.md | 8 + 3 files changed, 18 insertions(+), 358 deletions(-) create mode 100644 egs/yesno/ASR/tdnn/README.md diff --git a/egs/librispeech/ASR/conformer_ctc/README.md b/egs/librispeech/ASR/conformer_ctc/README.md index 130d21351..0092fd14e 100644 --- a/egs/librispeech/ASR/conformer_ctc/README.md +++ b/egs/librispeech/ASR/conformer_ctc/README.md @@ -1,351 +1,4 @@ -# How to use a pre-trained model to transcribe a sound file or multiple sound files - -(See the bottom of this document for the link to a colab notebook.) - -You need to prepare 4 files: - - - a model checkpoint file, e.g., epoch-20.pt - - HLG.pt, the decoding graph - - words.txt, the word symbol table - - a sound file, whose sampling rate has to be 16 kHz. - Supported formats are those supported by `torchaudio.load()`, - e.g., wav and flac. - -Also, you need to install `kaldifeat`. Please refer to - for installation. - -```bash -./conformer_ctc/pretrained.py --help -``` - -displays the help information. - -## HLG decoding - -Once you have the above files ready and have `kaldifeat` installed, -you can run: - -```bash -./conformer_ctc/pretrained.py \ - --checkpoint /path/to/your/checkpoint.pt \ - --words-file /path/to/words.txt \ - --HLG /path/to/HLG.pt \ - /path/to/your/sound.wav -``` - -and you will see the transcribed result. - -If you want to transcribe multiple files at the same time, you can use: - -```bash -./conformer_ctc/pretrained.py \ - --checkpoint /path/to/your/checkpoint.pt \ - --words-file /path/to/words.txt \ - --HLG /path/to/HLG.pt \ - /path/to/your/sound1.wav \ - /path/to/your/sound2.wav \ - /path/to/your/sound3.wav -``` - -**Note**: This is the fastest decoding method. - -## HLG decoding + LM rescoring - -`./conformer_ctc/pretrained.py` also supports `whole lattice LM rescoring` -and `attention decoder rescoring`. - -To use whole lattice LM rescoring, you also need the following files: - - - G.pt, e.g., `data/lm/G_4_gram.pt` if you have run `./prepare.sh` - -The command to run decoding with LM rescoring is: - -```bash -./conformer_ctc/pretrained.py \ - --checkpoint /path/to/your/checkpoint.pt \ - --words-file /path/to/words.txt \ - --HLG /path/to/HLG.pt \ - --method whole-lattice-rescoring \ - --G data/lm/G_4_gram.pt \ - --ngram-lm-scale 0.8 \ - /path/to/your/sound1.wav \ - /path/to/your/sound2.wav \ - /path/to/your/sound3.wav -``` - -## HLG Decoding + LM rescoring + attention decoder rescoring - -To use attention decoder for rescoring, you need the following extra information: - - - sos token ID - - eos token ID - -The command to run decoding with attention decoder rescoring is: - -```bash -./conformer_ctc/pretrained.py \ - --checkpoint /path/to/your/checkpoint.pt \ - --words-file /path/to/words.txt \ - --HLG /path/to/HLG.pt \ - --method attention-decoder \ - --G data/lm/G_4_gram.pt \ - --ngram-lm-scale 1.3 \ - --attention-decoder-scale 1.2 \ - --lattice-score-scale 0.5 \ - --num-paths 100 \ - --sos-id 1 \ - --eos-id 1 \ - /path/to/your/sound1.wav \ - /path/to/your/sound2.wav \ - /path/to/your/sound3.wav -``` - -# Decoding with a pre-trained model in action - -We have uploaded a pre-trained model to - -The following shows the steps about the usage of the provided pre-trained model. - -### (1) Download the pre-trained model - -```bash -sudo apt-get install git-lfs -cd /path/to/icefall/egs/librispeech/ASR -git lfs install -mkdir tmp -cd tmp -git clone https://huggingface.co/pkufool/conformer_ctc -``` - -**CAUTION**: You have to install `git-lfst` to download the pre-trained model. - -You will find the following files: - -``` -tmp -`-- conformer_ctc - |-- README.md - |-- data - | |-- lang_bpe - | | |-- HLG.pt - | | |-- bpe.model - | | |-- tokens.txt - | | `-- words.txt - | `-- lm - | `-- G_4_gram.pt - |-- exp - | `-- pretraind.pt - `-- test_wavs - |-- 1089-134686-0001.flac - |-- 1221-135766-0001.flac - |-- 1221-135766-0002.flac - `-- trans.txt - -6 directories, 11 files -``` - -**File descriptions**: - - - `data/lang_bpe/HLG.pt` - - It is the decoding graph. - - - `data/lang_bpe/bpe.model` - - It is a sentencepiece model. You can use it to reproduce our results. - - - `data/lang_bpe/tokens.txt` - - It contains tokens and their IDs, generated from `bpe.model`. - Provided only for convienice so that you can look up the SOS/EOS ID easily. - - - `data/lang_bpe/words.txt` - - It contains words and their IDs. - - - `data/lm/G_4_gram.pt` - - It is a 4-gram LM, useful for LM rescoring. - - - `exp/pretrained.pt` - - It contains pre-trained model parameters, obtained by averaging - checkpoints from `epoch-15.pt` to `epoch-34.pt`. - Note: We have removed optimizer `state_dict` to reduce file size. - - - `test_waves/*.flac` - - It contains some test sound files from LibriSpeech `test-clean` dataset. - - - `test_waves/trans.txt` - - It contains the reference transcripts for the sound files in `test_waves/`. - -The information of the test sound files is listed below: - -``` -$ soxi tmp/conformer_ctc/test_wavs/*.flac - -Input File : 'tmp/conformer_ctc/test_wavs/1089-134686-0001.flac' -Channels : 1 -Sample Rate : 16000 -Precision : 16-bit -Duration : 00:00:06.62 = 106000 samples ~ 496.875 CDDA sectors -File Size : 116k -Bit Rate : 140k -Sample Encoding: 16-bit FLAC - -Input File : 'tmp/conformer_ctc/test_wavs/1221-135766-0001.flac' -Channels : 1 -Sample Rate : 16000 -Precision : 16-bit -Duration : 00:00:16.71 = 267440 samples ~ 1253.62 CDDA sectors -File Size : 343k -Bit Rate : 164k -Sample Encoding: 16-bit FLAC - -Input File : 'tmp/conformer_ctc/test_wavs/1221-135766-0002.flac' -Channels : 1 -Sample Rate : 16000 -Precision : 16-bit -Duration : 00:00:04.83 = 77200 samples ~ 361.875 CDDA sectors -File Size : 105k -Bit Rate : 174k -Sample Encoding: 16-bit FLAC - -Total Duration of 3 files: 00:00:28.16 -``` - -### (2) Use HLG decoding - -```bash -cd /path/to/icefall/egs/librispeech/ASR - -./conformer_ctc/pretrained.py \ - --checkpoint ./tmp/conformer_ctc/exp/pretraind.pt \ - --words-file ./tmp/conformer_ctc/data/lang_bpe/words.txt \ - --HLG ./tmp/conformer_ctc/data/lang_bpe/HLG.pt \ - ./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac \ - ./tmp/conformer_ctc/test_wavs/1221-135766-0001.flac \ - ./tmp/conformer_ctc/test_wavs/1221-135766-0002.flac -``` - -The output is given below: - -``` -2021-08-20 11:03:05,712 INFO [pretrained.py:217] device: cuda:0 -2021-08-20 11:03:05,712 INFO [pretrained.py:219] Creating model -2021-08-20 11:03:11,345 INFO [pretrained.py:238] Loading HLG from ./tmp/conformer_ctc/data/lang_bpe/HLG.pt -2021-08-20 11:03:18,442 INFO [pretrained.py:255] Constructing Fbank computer -2021-08-20 11:03:18,444 INFO [pretrained.py:265] Reading sound files: ['./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac', './tmp/conformer_ctc/test_wavs/1221-135766-0001.flac', './tmp/conformer_ctc/test_wavs/1221-135766-0002.flac'] -2021-08-20 11:03:18,507 INFO [pretrained.py:271] Decoding started -2021-08-20 11:03:18,795 INFO [pretrained.py:300] Use HLG decoding -2021-08-20 11:03:19,149 INFO [pretrained.py:339] -./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac: -AFTER EARLY NIGHTFALL THE YELLOW LAMPS WOULD LIGHT UP HERE AND THERE THE SQUALID QUARTER OF THE BROTHELS - -./tmp/conformer_ctc/test_wavs/1221-135766-0001.flac: -GOD AS A DIRECT CONSEQUENCE OF THE SIN WHICH MAN THUS PUNISHED HAD GIVEN HER A LOVELY CHILD WHOSE PLACE WAS ON THAT SAME DISHONOURED -BOSOM TO CONNECT HER PARENT FOR EVER WITH THE RACE AND DESCENT OF MORTALS AND TO BE FINALLY A BLESSED SOUL IN HEAVEN - -./tmp/conformer_ctc/test_wavs/1221-135766-0002.flac: -YET THESE THOUGHTS AFFECTED HESTER PRYNNE LESS WITH HOPE THAN APPREHENSION - - -2021-08-20 11:03:19,149 INFO [pretrained.py:341] Decoding Done -``` - -### (3) Use HLG decoding + LM rescoring - -```bash -./conformer_ctc/pretrained.py \ - --checkpoint ./tmp/conformer_ctc/exp/pretraind.pt \ - --words-file ./tmp/conformer_ctc/data/lang_bpe/words.txt \ - --HLG ./tmp/conformer_ctc/data/lang_bpe/HLG.pt \ - --method whole-lattice-rescoring \ - --G ./tmp/conformer_ctc/data/lm/G_4_gram.pt \ - --ngram-lm-scale 0.8 \ - ./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac \ - ./tmp/conformer_ctc/test_wavs/1221-135766-0001.flac \ - ./tmp/conformer_ctc/test_wavs/1221-135766-0002.flac -``` - -The output is: - -``` -2021-08-20 11:12:17,565 INFO [pretrained.py:217] device: cuda:0 -2021-08-20 11:12:17,565 INFO [pretrained.py:219] Creating model -2021-08-20 11:12:23,728 INFO [pretrained.py:238] Loading HLG from ./tmp/conformer_ctc/data/lang_bpe/HLG.pt -2021-08-20 11:12:30,035 INFO [pretrained.py:246] Loading G from ./tmp/conformer_ctc/data/lm/G_4_gram.pt -2021-08-20 11:13:10,779 INFO [pretrained.py:255] Constructing Fbank computer -2021-08-20 11:13:10,787 INFO [pretrained.py:265] Reading sound files: ['./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac', './tmp/conformer_ctc/test_wavs/1221-135766-0001.flac', './tmp/conformer_ctc/test_wavs/1221-135766-0002.flac'] -2021-08-20 11:13:10,798 INFO [pretrained.py:271] Decoding started -2021-08-20 11:13:11,085 INFO [pretrained.py:305] Use HLG decoding + LM rescoring -2021-08-20 11:13:11,736 INFO [pretrained.py:339] -./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac: -AFTER EARLY NIGHTFALL THE YELLOW LAMPS WOULD LIGHT UP HERE AND THERE THE SQUALID QUARTER OF THE BROTHELS - -./tmp/conformer_ctc/test_wavs/1221-135766-0001.flac: -GOD AS A DIRECT CONSEQUENCE OF THE SIN WHICH MAN THUS PUNISHED HAD GIVEN HER A LOVELY CHILD WHOSE PLACE WAS ON THAT SAME DISHONOURED -BOSOM TO CONNECT HER PARENT FOR EVER WITH THE RACE AND DESCENT OF MORTALS AND TO BE FINALLY A BLESSED SOUL IN HEAVEN - -./tmp/conformer_ctc/test_wavs/1221-135766-0002.flac: -YET THESE THOUGHTS AFFECTED HESTER PRYNNE LESS WITH HOPE THAN APPREHENSION - - -2021-08-20 11:13:11,737 INFO [pretrained.py:341] Decoding Done -``` - -### (4) Use HLG decoding + LM rescoring + attention decoder rescoring - -```bash -./conformer_ctc/pretrained.py \ - --checkpoint ./tmp/conformer_ctc/exp/pretraind.pt \ - --words-file ./tmp/conformer_ctc/data/lang_bpe/words.txt \ - --HLG ./tmp/conformer_ctc/data/lang_bpe/HLG.pt \ - --method attention-decoder \ - --G ./tmp/conformer_ctc/data/lm/G_4_gram.pt \ - --ngram-lm-scale 1.3 \ - --attention-decoder-scale 1.2 \ - --lattice-score-scale 0.5 \ - --num-paths 100 \ - --sos-id 1 \ - --eos-id 1 \ - ./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac \ - ./tmp/conformer_ctc/test_wavs/1221-135766-0001.flac \ - ./tmp/conformer_ctc/test_wavs/1221-135766-0002.flac -``` - -The output is: - -``` -2021-08-20 11:19:11,397 INFO [pretrained.py:217] device: cuda:0 -2021-08-20 11:19:11,397 INFO [pretrained.py:219] Creating model -2021-08-20 11:19:17,354 INFO [pretrained.py:238] Loading HLG from ./tmp/conformer_ctc/data/lang_bpe/HLG.pt -2021-08-20 11:19:24,615 INFO [pretrained.py:246] Loading G from ./tmp/conformer_ctc/data/lm/G_4_gram.pt -2021-08-20 11:20:04,576 INFO [pretrained.py:255] Constructing Fbank computer -2021-08-20 11:20:04,584 INFO [pretrained.py:265] Reading sound files: ['./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac', './tmp/conformer_ctc/test_wavs/1221-135766-0001.flac', './tmp/conformer_ctc/test_wavs/1221-135766-0002.flac'] -2021-08-20 11:20:04,595 INFO [pretrained.py:271] Decoding started -2021-08-20 11:20:04,854 INFO [pretrained.py:313] Use HLG + LM rescoring + attention decoder rescoring -2021-08-20 11:20:05,805 INFO [pretrained.py:339] -./tmp/conformer_ctc/test_wavs/1089-134686-0001.flac: -AFTER EARLY NIGHTFALL THE YELLOW LAMPS WOULD LIGHT UP HERE AND THERE THE SQUALID QUARTER OF THE BROTHELS - -./tmp/conformer_ctc/test_wavs/1221-135766-0001.flac: -GOD AS A DIRECT CONSEQUENCE OF THE SIN WHICH MAN THUS PUNISHED HAD GIVEN HER A LOVELY CHILD WHOSE PLACE WAS ON THAT SAME DISHONOURED -BOSOM TO CONNECT HER PARENT FOR EVER WITH THE RACE AND DESCENT OF MORTALS AND TO BE FINALLY A BLESSED SOUL IN HEAVEN - -./tmp/conformer_ctc/test_wavs/1221-135766-0002.flac: -YET THESE THOUGHTS AFFECTED HESTER PRYNNE LESS WITH HOPE THAN APPREHENSION - - -2021-08-20 11:20:05,805 INFO [pretrained.py:341] Decoding Done -``` - -**NOTE**: We provide a colab notebook for demonstration. -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1huyupXAcHsUrKaWfI83iMEJ6J0Nh0213?usp=sharing) - -Due to limited memory provided by Colab, you have to upgrade to Colab Pro to -run `HLG decoding + LM rescoring` and `HLG decoding + LM rescoring + attention decoder rescoring`. -Otherwise, you can only run `HLG decoding` with Colab. +Please visit + +for how to run this recipe. diff --git a/egs/yesno/ASR/README.md b/egs/yesno/ASR/README.md index 653c576fa..6db2f782f 100644 --- a/egs/yesno/ASR/README.md +++ b/egs/yesno/ASR/README.md @@ -1,15 +1,14 @@ ## Yesno recipe -You can run the recipe with **CPU**. +This is the simplest ASR recipe in `icefall`. - -[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1tIjjzaJc3IvGyKiMCDWO-TSnBgkcuN3B?usp=sharing) - -The above Colab notebook finishes the training using **CPU** -within two minutes (50 epochs in total). - -The WER is +It can be run on CPU and takes less than 30 seconds to +get the following WER: ``` [test_set] %WER 0.42% [1 / 240, 0 ins, 1 del, 0 sub ] ``` + +Please refer to + +for detailed instructions. diff --git a/egs/yesno/ASR/tdnn/README.md b/egs/yesno/ASR/tdnn/README.md new file mode 100644 index 000000000..49722a779 --- /dev/null +++ b/egs/yesno/ASR/tdnn/README.md @@ -0,0 +1,8 @@ + +## How to run this recipe + +You can find detailed instructions by visiting + + +It describes how to run this recipe and how to use +a pre-trained model with `./pretrained.py`.