diff --git a/.buildinfo b/.buildinfo index d727d6619..46f8d58fb 100644 --- a/.buildinfo +++ b/.buildinfo @@ -1,4 +1,4 @@ # Sphinx build info version 1 # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. -config: 3ca2e66d59e42ffdb5e0a5ba2153f99e +config: cfc3e6ecc44ed7573f700065af8738a7 tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/_sources/recipes/librispeech/lstm_pruned_stateless_transducer.rst.txt b/_sources/recipes/librispeech/lstm_pruned_stateless_transducer.rst.txt index 0aeccb70a..b9d5bdcba 100644 --- a/_sources/recipes/librispeech/lstm_pruned_stateless_transducer.rst.txt +++ b/_sources/recipes/librispeech/lstm_pruned_stateless_transducer.rst.txt @@ -1,5 +1,5 @@ -Transducer -========== +LSTM Transducer +=============== .. hint:: @@ -7,7 +7,7 @@ Transducer for pretrained models if you don't want to train a model from scratch. -This tutorial shows you how to train a transducer model +This tutorial shows you how to train an LSTM transducer model with the `LibriSpeech `_ dataset. We use pruned RNN-T to compute the loss. @@ -20,9 +20,9 @@ We use pruned RNN-T to compute the loss. The transducer model consists of 3 parts: - - Encoder, a.k.a, transcriber. We use an LSTM model - - Decoder, a.k.a, predictor. We use a model consisting of ``nn.Embedding`` - and ``nn.Conv1d`` + - Encoder, a.k.a, the transcription network. We use an LSTM model + - Decoder, a.k.a, the prediction network. We use a stateless model consisting of + ``nn.Embedding`` and ``nn.Conv1d`` - Joiner, a.k.a, the joint network. .. caution:: @@ -74,7 +74,11 @@ Data preparation The script ``./prepare.sh`` handles the data preparation for you, **automagically**. All you need to do is to run it. -The data preparation contains several stages, you can use the following two +.. note:: + + We encourage you to read ``./prepare.sh``. + +The data preparation contains several stages. You can use the following two options: - ``--stage`` @@ -263,7 +267,7 @@ You will find the following files in that directory: - ``tensorboard/`` - This folder contains TensorBoard logs. Training loss, validation loss, learning + This folder contains tensorBoard logs. Training loss, validation loss, learning rate, etc, are recorded in these logs. You can visualize them by: .. code-block:: bash @@ -287,7 +291,7 @@ You will find the following files in that directory: [2022-09-20T15:53:02] Total uploaded: 210171 scalars, 0 tensors, 0 binary objects Listening for new data in logdir... - Note there is a URL in the above output, click it and you will see + Note there is a URL in the above output. Click it and you will see the following screenshot: .. figure:: images/librispeech-lstm-transducer-tensorboard-log.png @@ -422,7 +426,7 @@ The following shows two examples: Export models ------------- -`lstm_transducer_stateless2/export.py `_ supports to export checkpoints from ``lstm_transducer_stateless2/exp`` in the following ways. +`lstm_transducer_stateless2/export.py `_ supports exporting checkpoints from ``lstm_transducer_stateless2/exp`` in the following ways. Export ``model.state_dict()`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -458,7 +462,7 @@ It will generate a file ``./lstm_transducer_stateless2/exp/pretrained.pt``. cd lstm_transducer_stateless2/exp ln -s pretrained epoch-9999.pt - And then pass `--epoch 9999 --avg 1 --use-averaged-model 0` to + And then pass ``--epoch 9999 --avg 1 --use-averaged-model 0`` to ``./lstm_transducer_stateless2/decode.py``. To use the exported model with ``./lstm_transducer_stateless2/pretrained.py``, you @@ -506,6 +510,11 @@ To use the generated files with ``./lstm_transducer_stateless2/jit_pretrained``: /path/to/foo.wav \ /path/to/bar.wav +.. hint:: + + Please see ``_ + for how to use the exported models in ``sherpa``. + Export model for ncnn ~~~~~~~~~~~~~~~~~~~~~ @@ -576,37 +585,37 @@ It will generate the following files: - ``./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.param`` - ``./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.bin`` -To use the above generate files, run: +To use the above generated files, run: .. code-block:: bash -./lstm_transducer_stateless2/ncnn-decode.py \ - --bpe-model-filename ./data/lang_bpe_500/bpe.model \ - --encoder-param-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.param \ - --encoder-bin-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.bin \ - --decoder-param-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.param \ - --decoder-bin-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.bin \ - --joiner-param-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.param \ - --joiner-bin-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.bin \ - /path/to/foo.wav + ./lstm_transducer_stateless2/ncnn-decode.py \ + --bpe-model-filename ./data/lang_bpe_500/bpe.model \ + --encoder-param-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.param \ + --encoder-bin-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.bin \ + --decoder-param-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.param \ + --decoder-bin-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.bin \ + --joiner-param-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.param \ + --joiner-bin-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.bin \ + /path/to/foo.wav .. code-block:: bash -./lstm_transducer_stateless2/streaming-ncnn-decode.py \ - --bpe-model-filename ./data/lang_bpe_500/bpe.model \ - --encoder-param-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.param \ - --encoder-bin-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.bin \ - --decoder-param-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.param \ - --decoder-bin-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.bin \ - --joiner-param-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.param \ - --joiner-bin-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.bin \ - /path/to/foo.wav + ./lstm_transducer_stateless2/streaming-ncnn-decode.py \ + --bpe-model-filename ./data/lang_bpe_500/bpe.model \ + --encoder-param-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.param \ + --encoder-bin-filename ./lstm_transducer_stateless2/exp/encoder_jit_trace-pnnx.ncnn.bin \ + --decoder-param-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.param \ + --decoder-bin-filename ./lstm_transducer_stateless2/exp/decoder_jit_trace-pnnx.ncnn.bin \ + --joiner-param-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.param \ + --joiner-bin-filename ./lstm_transducer_stateless2/exp/joiner_jit_trace-pnnx.ncnn.bin \ + /path/to/foo.wav To use the above generated files in C++, please see ``_ -It is able to generate a static linked library that can be run on Linux, Windows, -macOS, Raspberry Pi, etc. +It is able to generate a static linked executable that can be run on Linux, Windows, +macOS, Raspberry Pi, etc, without external dependencies. Download pretrained models -------------------------- diff --git a/objects.inv b/objects.inv index b0de8a293..622ff5f6a 100644 Binary files a/objects.inv and b/objects.inv differ diff --git a/recipes/index.html b/recipes/index.html index 8d6fd0780..e297f8c00 100644 --- a/recipes/index.html +++ b/recipes/index.html @@ -93,7 +93,7 @@ Currently, only speech recognition recipes are provided.

  • LibriSpeech
  • TIMIT
      diff --git a/recipes/librispeech/conformer_ctc.html b/recipes/librispeech/conformer_ctc.html index b3dc3df78..41f0824a5 100644 --- a/recipes/librispeech/conformer_ctc.html +++ b/recipes/librispeech/conformer_ctc.html @@ -19,7 +19,7 @@ - + @@ -54,7 +54,7 @@
    • Deployment with C++
    • -
  • Transducer
    • +
  • LSTM Transducer
  • TIMIT
    • @@ -1087,7 +1087,7 @@ Please see - +
    diff --git a/recipes/librispeech/index.html b/recipes/librispeech/index.html index 1e8ffe6a2..3cf6bb283 100644 --- a/recipes/librispeech/index.html +++ b/recipes/librispeech/index.html @@ -46,7 +46,7 @@
  • LibriSpeech
  • TIMIT
    • @@ -88,7 +88,7 @@ diff --git a/recipes/librispeech/lstm_pruned_stateless_transducer.html b/recipes/librispeech/lstm_pruned_stateless_transducer.html index 0f2bb88a6..6e59b9a85 100644 --- a/recipes/librispeech/lstm_pruned_stateless_transducer.html +++ b/recipes/librispeech/lstm_pruned_stateless_transducer.html @@ -4,7 +4,7 @@ - Transducer — icefall 0.1 documentation + LSTM Transducer — icefall 0.1 documentation