From 3b5acd16e9ad9354ef0646fbf077d9914cb78cc2 Mon Sep 17 00:00:00 2001 From: Guanbo Wang Date: Sat, 14 May 2022 20:27:32 -0400 Subject: [PATCH] Update results --- README.md | 6 +++--- egs/gigaspeech/ASR/README.md | 2 +- egs/gigaspeech/ASR/RESULTS.md | 29 ++++++++++++++--------------- 3 files changed, 18 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index c4dad6aaf..94ef9d1d6 100644 --- a/README.md +++ b/README.md @@ -210,9 +210,9 @@ We provide a Colab notebook to run a pre-trained Pruned Transducer Stateless mod | | Dev | Test | |----------------------|-------|-------| -| greedy search | 10.59 | 10.87 | -| fast beam search | 10.56 | 10.80 | -| modified beam search | 10.52 | 10.62 | +| greedy search | 10.51 | 10.73 | +| fast beam search | 10.50 | 10.69 | +| modified beam search | 10.40 | 10.51 | ## Deployment with C++ diff --git a/egs/gigaspeech/ASR/README.md b/egs/gigaspeech/ASR/README.md index 1fca69e8b..32a0457c6 100644 --- a/egs/gigaspeech/ASR/README.md +++ b/egs/gigaspeech/ASR/README.md @@ -16,6 +16,6 @@ ln -sfv /path/to/GigaSpeech download/GigaSpeech | | Dev | Test | |--------------------------------|-------|-------| | `conformer_ctc` | 10.47 | 10.58 | -| `pruned_transducer_stateless2` | 10.52 | 10.62 | +| `pruned_transducer_stateless2` | 10.40 | 10.51 | See [RESULTS](/egs/gigaspeech/ASR/RESULTS.md) for details. diff --git a/egs/gigaspeech/ASR/RESULTS.md b/egs/gigaspeech/ASR/RESULTS.md index de7b84202..7ab565844 100644 --- a/egs/gigaspeech/ASR/RESULTS.md +++ b/egs/gigaspeech/ASR/RESULTS.md @@ -11,13 +11,15 @@ decoder contains only an embedding layer, a Conv1d (with kernel size 2) and a linear layer (to transform tensor dim). k2 pruned RNN-T loss is used. +The best WER, as of 2022-05-12, for the gigaspeech is below + Results are: | | Dev | Test | |----------------------|-------|-------| -| greedy search | 10.59 | 10.87 | -| fast beam search | 10.56 | 10.80 | -| modified beam search | 10.52 | 10.62 | +| greedy search | 10.51 | 10.73 | +| fast beam search | 10.50 | 10.69 | +| modified beam search | 10.40 | 10.51 | To reproduce the above result, use the following commands for training: @@ -39,33 +41,30 @@ and the following commands for decoding: ```bash # greedy search ./pruned_transducer_stateless2/decode.py \ - --epoch 29 \ - --avg 11 \ + --iter 3488000 \ + --avg 20 \ --decoding-method greedy_search \ --exp-dir pruned_transducer_stateless2/exp \ --bpe-model data/lang_bpe_500/bpe.model \ - --max-duration 20 \ - --num-workers 1 + --max-duration 600 # fast beam search ./pruned_transducer_stateless2/decode.py \ - --epoch 29 \ - --avg 9 \ + --iter 3488000 \ + --avg 20 \ --decoding-method fast_beam_search \ --exp-dir pruned_transducer_stateless2/exp \ --bpe-model data/lang_bpe_500/bpe.model \ - --max-duration 20 \ - --num-workers 1 + --max-duration 600 # modified beam search ./pruned_transducer_stateless2/decode.py \ - --epoch 29 \ - --avg 8 \ + --iter 3488000 \ + --avg 15 \ --decoding-method modified_beam_search \ --exp-dir pruned_transducer_stateless2/exp \ --bpe-model data/lang_bpe_500/bpe.model \ - --max-duration 20 \ - --num-workers 1 + --max-duration 600 ``` Pretrained model is available at