Teo Wen Shen
da87e7fc99
add weights_only=False to torch.load ( #1984 )
2025-07-10 15:27:08 +08:00
Mistmoon
9293edc62f
Add cr-ctc loss and ctc-decode in aishell ( #1980 )
2025-07-08 14:47:24 +08:00
Fangjun Kuang
fba5e67d5e
Fix CI tests. ( #1974 )
...
- Introduce unified AMP helpers (create_grad_scaler, torch_autocast) to handle
deprecations in PyTorch ≥2.3.0
- Replace direct uses of torch.cuda.amp.GradScaler and torch.cuda.amp.autocast
with the new utilities across all training and inference scripts
- Update all torch.load calls to include weights_only=False for compatibility with
newer PyTorch versions
2025-07-01 13:47:55 +08:00
Wei Kang
f80a2ee110
Decrease num_buckets & remove shuffle_buffer_size ( #1955 )
2025-06-19 12:26:37 +08:00
Fangjun Kuang
300a821f58
Fix aishell training ( #1916 )
2025-04-10 10:30:37 +08:00
Fangjun Kuang
d4d4f281ec
Revert "Replace deprecated pytorch methods ( #1814 )" ( #1841 )
...
This reverts commit 3e4da5f78160d3dba3bdf97968bd7ceb8c11631f.
2024-12-18 16:49:57 +08:00
Li Peng
3e4da5f781
Replace deprecated pytorch methods ( #1814 )
...
* Replace deprecated pytorch methods
- torch.cuda.amp.GradScaler(...) => torch.amp.GradScaler("cuda", ...)
- torch.cuda.amp.autocast(...) => torch.amp.autocast("cuda", ...)
* Replace `with autocast(...)` with `with autocast("cuda", ...)`
Co-authored-by: Li Peng <lipeng@unisound.ai>
2024-12-16 10:24:16 +08:00
Zengwei Yao
334beed2af
fix usages of returned losses after adding attention-decoder in zipformer ( #1689 )
2024-07-12 16:50:58 +08:00
Fangjun Kuang
3059eb4511
Fix doc URLs ( #1660 )
2024-06-21 11:10:14 +08:00
Wei Kang
b49351fc39
Update README.md for conformer-ctc ( #1609 )
2024-04-28 09:56:13 +08:00
Dadoou
6cbddaa8e3
Add base choice to model_name argument for whisper model. ( #1573 )
...
Co-authored-by: dadoou <dadoou@yandex.com>
2024-04-02 09:47:38 +08:00
zr_jin
9b0eae3b4a
fixes for init value of diagnostics.TensorDiagnosticOptions ( #1555 )
2024-03-18 17:14:29 +08:00
zr_jin
bf2f94346c
Enabling char_level and compute_CER for aishell recipe ( #1554 )
...
* init fix
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2024-03-18 11:57:47 +08:00
zr_jin
eb132da00d
additional instruction for the grad_scale is too small error ( #1550 )
2024-03-14 11:33:49 +08:00
zr_jin
c3f6f28116
Zipformer recipe for Cantonese dataset MDCC ( #1537 )
...
* init commit
* Create README.md
* handle code switching cases
* misc. fixes
* added manifest statistics
* init commit for the zipformer recipe
* added scripts for exporting model
* added RESULTS.md
* added scripts for streaming related stuff
* doc str fixed
2024-03-13 10:01:28 +08:00
Fangjun Kuang
60986c3ac1
Fix default value for --context-size in icefall. ( #1538 )
2024-03-08 20:47:13 +08:00
Yuekai Zhang
5df24c1685
Whisper large fine-tuning on wenetspeech, mutli-hans-zh ( #1483 )
...
* add whisper fbank for wenetspeech
* add whisper fbank for other dataset
* add str to bool
* add decode for wenetspeech
* add requirments.txt
* add original model decode with 30s
* test feature extractor speed
* add aishell2 feat
* change compute feature batch
* fix overwrite
* fix executor
* regression
* add kaldifeatwhisper fbank
* fix io issue
* parallel jobs
* use multi machines
* add wenetspeech fine-tune scripts
* add monkey patch codes
* remove useless file
* fix subsampling factor
* fix too long audios
* add remove long short
* fix whisper version to support multi batch beam
* decode all wav files
* remove utterance more than 30s in test_net
* only test net
* using soft links
* add kespeech whisper feats
* fix index error
* add manifests for whisper
* change to licomchunky writer
* add missing option
* decrease cpu usage
* add speed perturb for kespeech
* fix kespeech speed perturb
* add dataset
* load checkpoint from specific path
* add speechio
* add speechio results
---------
Co-authored-by: zr_jin <peter.jin.cn@gmail.com>
2024-03-07 19:04:27 +08:00
zr_jin
a813186f64
minor fix for docstr and default param. ( #1490 )
...
* Update train.py and README.md
2024-02-05 12:47:52 +08:00
zr_jin
37b975cac9
fixed a CI test for wenetspeech ( #1476 )
...
* Comply to issue #1149
https://github.com/k2-fsa/icefall/issues/1149
2024-01-27 06:41:56 +08:00
Yuekai Zhang
1c30847947
Whisper Fine-tuning Recipe on Aishell1 ( #1466 )
...
* add decode seamlessm4t
* add requirements
* add decoding with avg model
* add token files
* add custom tokenizer
* support deepspeed to finetune large model
* support large-v3
* add model saving
* using monkey patch to replace models
* add manifest dir option
2024-01-27 00:32:30 +08:00
Yifan Yang
5dfc3ed7f9
Fix buffer size of DynamicBucketingSampler ( #1468 )
...
* Fix buffer size
* Fix for flake8
---------
Co-authored-by: yifanyeung <yifanyeung@yifanyeung.local>
2024-01-21 02:10:42 +08:00
zr_jin
7bdde9174c
A Zipformer recipe with Byte-level BPE for Aishell-1 ( #1464 )
...
* init commit
* Update train.py
* Update decode.py
* Update RESULTS.md
* added `vocab_size`
* removed unused softlinks
* added scripts for testing pretrained models
* set `bpe_model` as required
* re-org the bbpe recipe for aishell
2024-01-16 21:08:35 +08:00
Karel Vesely
716b82cc3a
streaming_decode.py, relax the audio range from [-1,+1] to [-10,+10] ( #1448 )
...
- some AudioTransform classes produce audio signals out of range [-1,+1]
- Resample produced 1.0079
- The range [-10,+10] was chosen to still be able to reliably
distinguish from the [-32k,+32k] signal...
- this is related to : https://github.com/lhotse-speech/lhotse/issues/1254
2024-01-05 10:21:27 +08:00
Fangjun Kuang
8136ad775b
Use high_freq -400 in computing fbank features. ( #1447 )
...
See also https://github.com/k2-fsa/sherpa-onnx/issues/514
2024-01-04 13:59:32 +08:00
Fangjun Kuang
e9ec827de7
Rename zipformer2 to zipformer_for_ncnn_export_only to avoid confusion. ( #1407 )
2023-12-08 14:29:24 +08:00
Wei Kang
11d816d174
Add cumstomized score for hotwords ( #1385 )
...
* add custom score for each hotword
* Add more comments
* Fix deocde
* fix style
* minor fixes
2023-11-18 18:47:55 +08:00
Fangjun Kuang
666d69b20d
Rename train2.py to avoid confusion ( #1386 )
2023-11-17 18:12:59 +08:00
zr_jin
23913f6afd
Minor refinements for some stale but recently merged PRs ( #1354 )
...
* incorporate https://github.com/k2-fsa/icefall/pull/1269
* incorporate https://github.com/k2-fsa/icefall/pull/1301
* black formatted
* incorporate https://github.com/k2-fsa/icefall/pull/1162
* black formatted
2023-10-31 10:28:20 +08:00
zr_jin
1814bbb0e7
typo fixed ( #1334 )
2023-10-25 00:03:33 +08:00
zr_jin
d76c3fe472
Migrate zipformer model to other Chinese datasets ( #1216 )
...
added zipformer recipe for AISHELL-1
2023-10-24 16:24:46 +08:00
zr_jin
92ef561ff7
Minor fixes for torch.jit.script support ( #1329 )
2023-10-24 01:10:50 +08:00
zr_jin
d2bd0933b1
Compatibility with the latest Lhotse ( #1314 )
2023-10-17 21:22:32 +08:00
zr_jin
1ef349d120
[WIP] AISHELL-1 pruned transducer stateless7 streaming recipe ( #1300 )
...
* `pruned_transudcer_stateless7_streaming` for AISHELL-1
* Update train.py
* Update train2.py
* Update decode.py
* Update RESULTS.md
2023-10-16 16:28:16 +08:00
zr_jin
162ceaf4b3
fixes for data preparation ( #1307 )
...
Issue: #1306
2023-10-12 17:05:41 +08:00
zr_jin
0d09a44930
Update train.py ( #1299 )
2023-10-11 10:06:00 +08:00
Fangjun Kuang
f14b673408
Add HLG decoding with OpenFst on CPU for aishell conformer_ctc ( #1279 )
2023-10-01 13:46:16 +08:00
yaguang
8181d19860
check bbpe model exists in advance. ( #1277 )
2023-09-27 17:35:26 +08:00
yaguang
a5ba1133c4
Compatible with new lhotse versions. ( #1278 )
2023-09-27 17:33:38 +08:00
zr_jin
ef658d691e
fixes for init value of diagnostics.TensorDiagnosticOptions ( #1269 )
...
* fixes for `diagnostics`
Replace `2 ** 22` with `512` as the default value of `diagnostics.TensorDiagnosticOptions`
also black formatted some scripts
* fixed formatting issues
2023-09-24 17:06:47 +08:00
Fangjun Kuang
34e40a86b3
Fix exporting decoder model to onnx ( #1264 )
...
* Use torch.jit.script() to export the decoder model
See also https://github.com/k2-fsa/sherpa-onnx/issues/327
2023-09-22 09:57:15 +08:00
Fangjun Kuang
f5dc957d44
Fix CI tests ( #1266 )
2023-09-21 21:16:14 +08:00
zr_jin
7cc2dae940
Fixes to incorporate with the latest Lhotse release ( #1249 )
2023-09-13 12:39:49 +08:00
zr_jin
a81396b482
Use tokens.txt to replace bpe.model ( #1162 )
2023-08-12 16:53:59 +08:00
zr_jin
74806b744b
disable speed perturbation by default ( #1176 )
...
* disable speed perturbation by default
* minor fixes
* minor updates
* updated bash scripts to incorporate with the `speed-perturb` arg
* minor fixes
1. changed the naming scheme from `speed-perturb` to `perturb-speed` to align with the librispeech recipe
>> 00256a7669/egs/librispeech/ASR/local/compute_fbank_librispeech.py (L65)
2023-08-10 20:56:02 +08:00
zr_jin
856c0f2a60
fixed default param for an aishell recipe ( #1159 )
2023-07-04 19:12:39 +08:00
Wei Kang
219bba1310
zipformer wenetspeech ( #1130 )
...
* copy files
* update train.py
* small fixes
* Add decode.py
* Fix dataloader in decode.py
* add blank penalty
* Add blank-penalty to other decoding method
* Minor fixes
* add zipformer2 recipe
* Minor fixes
* Remove pruned7
* export and test models
* Replace bpe with tokens in export.py and pretrain.py
* Minor fixes
* Minor fixes
* Minor fixes
* Fix export
* Update results
* Fix zipformer-ctc
* Fix ci
* Fix ci
* Fix CI
* Fix CI
---------
Co-authored-by: Fangjun Kuang <csukuangfj@gmail.com>
2023-06-26 09:33:18 +08:00
Wei Kang
ba257efbcd
Add Context biasing ( #1038 )
...
* Add context biasing for librispeech
* Add context biasing for wenetspeech
* fix bugs
* Implement Aho-Corasick context graph
* fix some bugs
* Fixes to forward_one_step; add draw to context graph
* add output arc; fix black
* Fix wenetspeech tokenizer
* Minor fixes to the decode.py
2023-06-03 21:28:49 +08:00
Fangjun Kuang
7b0afbdc16
Remove cur_batch_idx ( #1102 )
2023-05-30 14:49:54 +08:00
marcoyang1998
585e7b224f
Aishell pruned_transducer_stateless7 ( #962 )
...
* Add pruned_transducer_stateless7 for Aishell
* update README.md
* update comments and small fixes
2023-05-23 11:04:33 +08:00
Wei Kang
80156dda09
Training with byte level BPE (AIShell) ( #986 )
...
* copy files from zipformer librispeech
* Add byte bpe training for aishell
* compile LG graph
* Support LG decoding
* Minor fixes
* black
* Minor fixes
* export & fix pretrain.py
* fix black
* Update RESULTS.md
* Fix export.py
2023-05-04 19:16:17 +08:00
