Export to ONNX
==============

In this section, we describe how to export models to ONNX.

.. hint::

  Only non-streaming conformer transducer models are tested.


When to use it
--------------

It you want to use an inference framework that supports ONNX
to run the pretrained model.


How to export
-------------

We use
`<https://github.com/k2-fsa/icefall/tree/master/egs/librispeech/ASR/pruned_transducer_stateless3>`_
as an example in the following.

.. code-block:: bash

    cd egs/librispeech/ASR
    epoch=14
    avg=2

    ./pruned_transducer_stateless3/export.py \
      --exp-dir ./pruned_transducer_stateless3/exp \
      --bpe-model data/lang_bpe_500/bpe.model \
      --epoch $epoch \
      --avg $avg \
      --onnx 1

It will generate the following files inside ``pruned_transducer_stateless3/exp``:

  - ``encoder.onnx``
  - ``decoder.onnx``
  - ``joiner.onnx``
  - ``joiner_encoder_proj.onnx``
  - ``joiner_decoder_proj.onnx``

You can use ``./pruned_transducer_stateless3/exp/onnx_pretrained.py`` to decode
waves with the generated files:

.. code-block:: bash

  ./pruned_transducer_stateless3/onnx_pretrained.py \
    --bpe-model ./data/lang_bpe_500/bpe.model \
    --encoder-model-filename ./pruned_transducer_stateless3/exp/encoder.onnx \
    --decoder-model-filename ./pruned_transducer_stateless3/exp/decoder.onnx \
    --joiner-model-filename ./pruned_transducer_stateless3/exp/joiner.onnx \
    --joiner-encoder-proj-model-filename ./pruned_transducer_stateless3/exp/joiner_encoder_proj.onnx \
    --joiner-decoder-proj-model-filename ./pruned_transducer_stateless3/exp/joiner_decoder_proj.onnx \
    /path/to/foo.wav \
    /path/to/bar.wav \
    /path/to/baz.wav


How to use the exported model
-----------------------------

We also provide `<https://github.com/k2-fsa/sherpa-onnx>`_
performing speech recognition using `onnxruntime <https://github.com/microsoft/onnxruntime>`_
with exported models.
It has been tested on Linux, macOS, and Windows.