Export to ONNX

In this section, we describe how to export models to ONNX.

Hint

Only non-streaming conformer transducer models are tested.

When to use it

It you want to use an inference framework that supports ONNX to run the pretrained model.

How to export

We use https://github.com/k2-fsa/icefall/tree/master/egs/librispeech/ASR/pruned_transducer_stateless3 as an example in the following.

cd egs/librispeech/ASR
epoch=14
avg=2

./pruned_transducer_stateless3/export.py \
  --exp-dir ./pruned_transducer_stateless3/exp \
  --bpe-model data/lang_bpe_500/bpe.model \
  --epoch $epoch \
  --avg $avg \
  --onnx 1

It will generate the following files inside pruned_transducer_stateless3/exp:

• encoder.onnx

• decoder.onnx

• joiner.onnx

• joiner_encoder_proj.onnx

• joiner_decoder_proj.onnx

You can use ./pruned_transducer_stateless3/exp/onnx_pretrained.py to decode waves with the generated files:

./pruned_transducer_stateless3/onnx_pretrained.py \
  --bpe-model ./data/lang_bpe_500/bpe.model \
  --encoder-model-filename ./pruned_transducer_stateless3/exp/encoder.onnx \
  --decoder-model-filename ./pruned_transducer_stateless3/exp/decoder.onnx \
  --joiner-model-filename ./pruned_transducer_stateless3/exp/joiner.onnx \
  --joiner-encoder-proj-model-filename ./pruned_transducer_stateless3/exp/joiner_encoder_proj.onnx \
  --joiner-decoder-proj-model-filename ./pruned_transducer_stateless3/exp/joiner_decoder_proj.onnx \
  /path/to/foo.wav \
  /path/to/bar.wav \
  /path/to/baz.wav

How to use the exported model

We also provide https://github.com/k2-fsa/sherpa-onnx performing speech recognition using onnxruntime with exported models. It has been tested on Linux, macOS, and Windows.