Fangjun Kuang
fba5e67d5e
Fix CI tests. ( #1974 )
...
- Introduce unified AMP helpers (create_grad_scaler, torch_autocast) to handle
deprecations in PyTorch ≥2.3.0
- Replace direct uses of torch.cuda.amp.GradScaler and torch.cuda.amp.autocast
with the new utilities across all training and inference scripts
- Update all torch.load calls to include weights_only=False for compatibility with
newer PyTorch versions
2025-07-01 13:47:55 +08:00
Fangjun Kuang
d4d4f281ec
Revert "Replace deprecated pytorch methods ( #1814 )" ( #1841 )
...
This reverts commit 3e4da5f78160d3dba3bdf97968bd7ceb8c11631f.
2024-12-18 16:49:57 +08:00
Li Peng
3e4da5f781
Replace deprecated pytorch methods ( #1814 )
...
* Replace deprecated pytorch methods
- torch.cuda.amp.GradScaler(...) => torch.amp.GradScaler("cuda", ...)
- torch.cuda.amp.autocast(...) => torch.amp.autocast("cuda", ...)
* Replace `with autocast(...)` with `with autocast("cuda", ...)`
Co-authored-by: Li Peng <lipeng@unisound.ai>
2024-12-16 10:24:16 +08:00
zr_jin
eb132da00d
additional instruction for the grad_scale is too small error ( #1550 )
2024-03-14 11:33:49 +08:00
zr_jin
242002e0bd
Strengthened style constraints ( #1527 )
2024-03-04 23:28:04 +08:00
Yifan Yang
5dfc3ed7f9
Fix buffer size of DynamicBucketingSampler ( #1468 )
...
* Fix buffer size
* Fix for flake8
---------
Co-authored-by: yifanyeung <yifanyeung@yifanyeung.local>
2024-01-21 02:10:42 +08:00
Fangjun Kuang
8136ad775b
Use high_freq -400 in computing fbank features. ( #1447 )
...
See also https://github.com/k2-fsa/sherpa-onnx/issues/514
2024-01-04 13:59:32 +08:00
Fangjun Kuang
79a42148db
Add CI test to cover zipformer/train.py ( #1424 )
2023-12-23 00:38:36 +08:00
Karel Vesely
543b4cc1ca
small enhanecements ( #1322 )
...
- add extra check of 'x' and 'x_lens' to earlier point in Transducer model
- specify 'utf' encoding when opening text files for writing (recogs,
errs)
2023-10-19 21:53:31 +08:00
marcoyang1998
52c24df61d
Fix model avg ( #1317 )
...
* fix a bug about the model_avg during finetuning by exchanging the order of loading pre-trained model and initializing avg model
* only match the exact module prefix
2023-10-18 17:36:14 +08:00
zr_jin
d2bd0933b1
Compatibility with the latest Lhotse ( #1314 )
2023-10-17 21:22:32 +08:00
Wen Ding
2b3c5d799f
Fix padding issues ( #1303 )
2023-10-11 16:58:00 +08:00
zr_jin
ef5da4824d
formatted the entire LibriSpeech recipe ( #1270 )
...
* formatted the entire librispeech recipe
* minor updates
2023-09-24 17:31:01 +08:00
zr_jin
ef658d691e
fixes for init value of diagnostics.TensorDiagnosticOptions ( #1269 )
...
* fixes for `diagnostics`
Replace `2 ** 22` with `512` as the default value of `diagnostics.TensorDiagnosticOptions`
also black formatted some scripts
* fixed formatting issues
2023-09-24 17:06:47 +08:00
Fangjun Kuang
34e40a86b3
Fix exporting decoder model to onnx ( #1264 )
...
* Use torch.jit.script() to export the decoder model
See also https://github.com/k2-fsa/sherpa-onnx/issues/327
2023-09-22 09:57:15 +08:00
Fangjun Kuang
f5dc957d44
Fix CI tests ( #1266 )
2023-09-21 21:16:14 +08:00
zr_jin
bbb03f7962
Update decoder.py ( #1262 )
2023-09-20 08:15:54 +08:00
zr_jin
7cc2dae940
Fixes to incorporate with the latest Lhotse release ( #1249 )
2023-09-13 12:39:49 +08:00
zr_jin
d50a9ea030
doc str fixes ( #1241 )
2023-09-07 16:34:53 +08:00
zr_jin
a81396b482
Use tokens.txt to replace bpe.model ( #1162 )
2023-08-12 16:53:59 +08:00
Fangjun Kuang
1dbbd7759e
Add tests for subsample.py and fix typos ( #1180 )
2023-07-25 14:46:18 +08:00
Peter Ross
b4c38d7547
Use symlinks for best epochs ( #1123 )
...
* utils: add symlink_or_copyfile
* pruned_transducer_stateless7: use symlinks (when possible) to output best epochs
* Rename function
---------
Co-authored-by: Yifan Yang <64255737+yfyeung@users.noreply.github.com>
2023-06-12 13:51:46 +08:00
Yifan Yang
ca60ced213
Fix typo ( #1114 )
...
* Fix typo for zipformer
* Fix typo for pruned_transducer_stateless7
* Fix typo for pruned_transducer_stateless7_ctc
* Fix typo for pruned_transducer_stateless7_ctc_bs
* Fix typo for pruned_transducer_stateless7_streaming
* Fix typo for pruned_transducer_stateless7_streaming_multi
* Fix file permissions for pruned_transducer_stateless7_streaming_multi
* Fix typo for pruned_transducer_stateless8
* Fix typo for pruned_transducer_stateless6
* Fix typo for pruned_transducer_stateless5
* Fix typo for pruned_transducer_stateless4
* Fix typo for pruned_transducer_stateless3
2023-06-02 14:12:42 +08:00
Yifan Yang
82f34a2388
Remove multidataset from librispeech/pruned_transducer_stateless7 ( #1105 )
...
* Add People's Speech to multidataset
* update
* remove multi from librispeech
2023-06-01 18:45:20 +08:00
Fangjun Kuang
7b0afbdc16
Remove cur_batch_idx ( #1102 )
2023-05-30 14:49:54 +08:00
Zengwei Yao
6826b076d4
add flops profiler, support for Zipformer encoder and Conformer encoder ( #1093 )
...
* add flops profiler, support for Zipformer encoder and Conformer encoder
* support for reworked conformer and old zipformer
* skip black check
2023-05-24 19:10:45 +08:00
Fangjun Kuang
dbcf0b41db
Fix stateless7 training error ( #1082 )
2023-05-23 12:52:02 +08:00
Yifan Yang
562bda91e4
Add adaption recipe for pruned_transducer_stateless7 ( #1059 )
...
* Add mux for finetune
* Add comments
* Fix for black
* Update finetune.py
2023-05-17 16:02:27 +08:00
Fangjun Kuang
5b50ffda54
support using mini librispeech in training ( #1048 )
...
* support mini librispeech in training
* update onnx export doc
2023-05-09 15:10:06 +08:00
Yifan Yang
2096e69bda
Use CutSet.mux for multidataset ( #1020 )
...
* Use CutSet.mux
* Remove mischange
* Fix for style check
2023-04-23 18:41:44 +08:00
Yifan Yang
d67a49afe4
Add multidataset ( #1010 )
...
* Add Common Voice for multidataset
* Add prepare_multidataset.sh
* Add dataset mixing
* Update prepare_multidataset.sh
* Update prepare_giga_speech.sh
* update comments
* Add split and shuffle mechanism
* Add multi-dataset train
* Fix for deleting
* Fix for modifying
* Add comments
* Change type for perturb_speed
* Fix for style check
* Small fix
* Add filter
* Remove warning
2023-04-21 18:09:41 +08:00
marcoyang1998
57d6482a79
Streaming Zipformer with multi-dataset ( #984 )
...
* modify train.py
* add right padding option in decode.py
* update RESULTS.md
2023-04-21 15:43:28 +08:00
Zengwei Yao
5f066d3d53
support decoding and computing RTF on test sets with onnx models ( #995 )
...
* support decode and compute RTF on test sets with onnx models
* support onnx export and decode in pruned_transducer_stateless
2023-04-12 19:04:50 +08:00
marcoyang1998
d337398d29
Shallow fusion for Aishell ( #954 )
...
* add shallow fusion and LODR for aishell
* update RESULTS
* add save by iterations
2023-04-03 16:20:29 +08:00
Yifan Yang
46bf6df62f
Remove simulate streaming from stateless7 ( #983 )
...
* Remove simulate streaming from stateless7
2023-04-03 14:55:45 +08:00
Fangjun Kuang
a632b24c35
Export int8 quantized models for non-streaming Zipformer. ( #977 )
...
* Export int8 quantized models for non-streaming Zipformer.
* Delete export-onnx.py
* Export int8 models for other folders
2023-03-31 22:46:19 +08:00
Zengwei Yao
bcc5923ab9
Support batch-wise forced-alignment ( #970 )
...
* support batch-wise forced-alignment based on beam search
* add length_norm to HypothesisList.topk()
* Use Hypothesis and HypothesisList instead
2023-03-28 23:24:24 +08:00
Fangjun Kuang
8c3ea93fc8
Save meta data to exported ONNX models ( #968 )
2023-03-27 11:39:29 +08:00
Zengwei Yao
7155769c19
minor fix, remove numel = p.numel() in optim.py ( #967 )
2023-03-24 15:30:29 +08:00
Peng He
f260a09ed4
remove if-branch at downsample pad in zipformer for onnx-export compatibility ( #965 )
2023-03-24 14:30:43 +08:00
marcoyang1998
7948624a22
Support fine-tuning ( #944 )
...
* support finetune
* add files for decoding giga
* support initializing modules
* add a fine-tune bash script
2023-03-17 13:44:29 +08:00
marcoyang1998
9ddd811925
Fix padding_idx ( #942 )
...
* fix padding_idx
* update RESULTS.md
2023-03-10 14:37:28 +08:00
Fangjun Kuang
f5de2e90c6
Fix style issues. ( #937 )
2023-03-08 22:56:04 +08:00
pehonnet
07243d136a
remove key from result filename ( #936 )
...
Co-authored-by: pe-honnet <pe.honnet@telepathy.ai>
2023-03-08 21:06:07 +08:00
Fangjun Kuang
c5e687ddf5
Export streaming zipformer to ncnn ( #906 )
2023-02-13 23:41:43 +08:00
Yifan Yang
c34ee67691
Update generate_model_from_checkpoint.py ( #901 )
2023-02-13 14:05:38 +08:00
Fangjun Kuang
2b995639b7
Add ONNX support for Zipformer and ConvEmformer ( #884 )
2023-02-09 00:02:38 +08:00
Yifan Yang
ffbf6d9199
Add generate_averaged_model.py ( #882 )
2023-02-07 16:19:08 +08:00
Zengwei Yao
1e6d6f8160
shuffle full Librispeech for zipformer recipes ( #869 )
...
* shuffle libri
2023-02-03 11:54:57 +08:00
Teo Wen Shen
1ce2bc1ee0
edit comments ( #852 )
2023-01-28 13:47:21 +08:00
