mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-09-04 06:34:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

add whisper fine-tuning results

Browse Source
This commit is contained in:
Yuekai Zhang 2024-01-17 16:17:32 +08:00
parent 557b35cefc
commit 84e4af93d7
1 changed files with 45 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
egs/aishell/ASR/RESULTS.md
View File

@ -1,5 +1,50 @@
## Results ## Results
### Aishell training results (Fine-tuning Pretrained Models)
#### Whisper
[./whisper](./whisper)
##### fine-tuning results on Aishell test set on whisper medium, large-v2, large-v3
| | test (before fine-tuning) | test (after fine-tuning) | comment |
|------------------------|------|------|-----------------------------------------|
| medium | 7.23 | 3.27 | --epoch 10 --avg 4, ddp |
| large-v2 | 6.56 | 2.47 | --epoch 10 --avg 6, deepspeed zero stage1 |
| large-v3 | 6.06 | 2.84 | --epoch 5 --avg 3, deepspeed zero stage1 |
Command for training is:
```bash
./prepare.sh --stage 30 --stop_stage 30
#fine-tuning with deepspeed zero stage 1
torchrun --nproc-per-node 8 ./whisper/train.py \
--max-duration 200 \
--use-fp16 1 \
--exp-dir whisper/exp_large_v2 \
--model-name large-v2 \
--deepspeed \
--deepspeed_config ./whisper/ds_config_zero1.json
# fine-tuning with ddp
torchrun --nproc-per-node 8 ./whisper/train.py \
--max-duration 200 \
--use-fp16 1 \
--exp-dir whisper/exp_medium \
--base-lr 1e-5 \
--model-name medium
```
Command for decoding is:
```bash
python3 ./whisper/decode.py \
--exp-dir whisper/exp_large_v2 \
--model-name large-v2 \
--epoch 999 --avg 1 \
--beam-size 10 --max-duration 50
```
Pretrained models, training logs, decoding logs, tensorboard and decoding results
are available at
<https://huggingface.co/yuekai/icefall_asr_aishell_whisper>
### Aishell training result (Stateless Transducer) ### Aishell training result (Stateless Transducer)
#### Zipformer (Non-streaming) #### Zipformer (Non-streaming)
@ -703,7 +748,6 @@ python3 ./transducer_stateless/decode.py \
--max-sym-per-frame 3 --max-sym-per-frame 3
``` ```
### Aishell training results (Transducer-stateless)
#### 2022-02-18 #### 2022-02-18
(Pingfeng Luo) : The tensorboard log for training is available at <https://tensorboard.dev/experiment/k3QL6QMhRbCwCKYKM9po9w/> (Pingfeng Luo) : The tensorboard log for training is available at <https://tensorboard.dev/experiment/k3QL6QMhRbCwCKYKM9po9w/>
And pretrained model is available at <https://huggingface.co/pfluo/icefall-aishell-transducer-stateless-char-2021-12-29> And pretrained model is available at <https://huggingface.co/pfluo/icefall-aishell-transducer-stateless-char-2021-12-29>