mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-08-08 09:32:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,216 Commits 5 Branches 3 Tags
Commit Graph

82 Commits

Author SHA1 Message Date
Fangjun Kuang
fba5e67d5e
Fix CI tests. (#1974) 
- Introduce unified AMP helpers (create_grad_scaler, torch_autocast) to handle 
  deprecations in PyTorch ≥2.3.0

- Replace direct uses of torch.cuda.amp.GradScaler and torch.cuda.amp.autocast 
  with the new utilities across all training and inference scripts

- Update all torch.load calls to include weights_only=False for compatibility with 
  newer PyTorch versions
 2025-07-01 13:47:55 +08:00
Fangjun Kuang
d4d4f281ec
Revert "Replace deprecated pytorch methods (#1814)" (#1841) 
This reverts commit 3e4da5f78160d3dba3bdf97968bd7ceb8c11631f.
 2024-12-18 16:49:57 +08:00
Li Peng
3e4da5f781
Replace deprecated pytorch methods (#1814) 
* Replace deprecated pytorch methods

- torch.cuda.amp.GradScaler(...) => torch.amp.GradScaler("cuda", ...)
- torch.cuda.amp.autocast(...) => torch.amp.autocast("cuda", ...)

* Replace `with autocast(...)` with `with autocast("cuda", ...)`


Co-authored-by: Li Peng <lipeng@unisound.ai>
 2024-12-16 10:24:16 +08:00
zr_jin
242002e0bd
Strengthened style constraints (#1527) 2024-03-04 23:28:04 +08:00
Yifan Yang
5dfc3ed7f9
Fix buffer size of DynamicBucketingSampler (#1468) 
* Fix buffer size

* Fix for flake8

---------

Co-authored-by: yifanyeung <yifanyeung@yifanyeung.local>
 2024-01-21 02:10:42 +08:00
Karel Vesely
716b82cc3a
streaming_decode.py, relax the audio range from [-1,+1] to [-10,+10] (#1448) 
- some AudioTransform classes produce audio signals out of range [-1,+1]
   - Resample produced 1.0079
   - The range [-10,+10] was chosen to still be able to reliably
     distinguish from the [-32k,+32k] signal...
- this is related to : https://github.com/lhotse-speech/lhotse/issues/1254
 2024-01-05 10:21:27 +08:00
Fangjun Kuang
8136ad775b
Use high_freq -400 in computing fbank features. (#1447) 
See also https://github.com/k2-fsa/sherpa-onnx/issues/514
 2024-01-04 13:59:32 +08:00
zr_jin
d2bd0933b1
Compatibility with the latest Lhotse (#1314) 2023-10-17 21:22:32 +08:00
Fangjun Kuang
34e40a86b3
Fix exporting decoder model to onnx (#1264) 
* Use torch.jit.script() to export the decoder model

See also https://github.com/k2-fsa/sherpa-onnx/issues/327
 2023-09-22 09:57:15 +08:00
Fangjun Kuang
f5dc957d44
Fix CI tests (#1266) 2023-09-21 21:16:14 +08:00
zr_jin
a81396b482
Use tokens.txt to replace bpe.model (#1162) 2023-08-12 16:53:59 +08:00
Fangjun Kuang
1dbbd7759e
Add tests for subsample.py and fix typos (#1180) 2023-07-25 14:46:18 +08:00
Yifan Yang
ca60ced213
Fix typo (#1114) 
* Fix typo for zipformer

* Fix typo for pruned_transducer_stateless7

* Fix typo for pruned_transducer_stateless7_ctc

* Fix typo for pruned_transducer_stateless7_ctc_bs

* Fix typo for pruned_transducer_stateless7_streaming

* Fix typo for pruned_transducer_stateless7_streaming_multi

* Fix file permissions for pruned_transducer_stateless7_streaming_multi

* Fix typo for pruned_transducer_stateless8

* Fix typo for pruned_transducer_stateless6

* Fix typo for pruned_transducer_stateless5

* Fix typo for pruned_transducer_stateless4

* Fix typo for pruned_transducer_stateless3
 2023-06-02 14:12:42 +08:00
Yifan Yang
d67a49afe4
Add multidataset (#1010) 
* Add Common Voice for multidataset

* Add prepare_multidataset.sh

* Add dataset mixing


* Update prepare_multidataset.sh

* Update prepare_giga_speech.sh

* update comments

* Add split and shuffle mechanism

* Add multi-dataset train

* Fix for deleting

* Fix for modifying

* Add comments

* Change type for perturb_speed

* Fix for style check

* Small fix

* Add filter

* Remove warning
 2023-04-21 18:09:41 +08:00
Zengwei Yao
5f066d3d53
support decoding and computing RTF on test sets with onnx models (#995) 
* support decode and compute RTF on test sets with onnx models

* support onnx export and decode in pruned_transducer_stateless
 2023-04-12 19:04:50 +08:00
Zengwei Yao
136aa94d57
remove duplicated lines (#988) 2023-04-06 17:47:33 +08:00
marcoyang1998
d337398d29
Shallow fusion for Aishell (#954) 
* add shallow fusion and LODR for aishell

* update RESULTS

* add save by iterations
 2023-04-03 16:20:29 +08:00
Fangjun Kuang
a632b24c35
Export int8 quantized models for non-streaming Zipformer. (#977) 
* Export int8 quantized models for non-streaming Zipformer.

* Delete export-onnx.py

* Export int8 models for other folders
 2023-03-31 22:46:19 +08:00
Zengwei Yao
2a5a75cb56
add option of using full attention for streaming model decoding (#975) 2023-03-30 14:30:13 +08:00
Fangjun Kuang
8c3ea93fc8
Save meta data to exported ONNX models (#968) 2023-03-27 11:39:29 +08:00
Fangjun Kuang
f5de2e90c6
Fix style issues. (#937) 2023-03-08 22:56:04 +08:00
pehonnet
07243d136a
remove key from result filename (#936) 
Co-authored-by: pe-honnet <pe.honnet@telepathy.ai>
 2023-03-08 21:06:07 +08:00
Fangjun Kuang
2b995639b7
Add ONNX support for Zipformer and ConvEmformer (#884) 2023-02-09 00:02:38 +08:00
Fangjun Kuang
7ae03f6c88
Add onnx export support for pruned_transducer_stateless5 (#883) 2023-02-07 17:47:08 +08:00
Fangjun Kuang
8d3810e289
Simplify ONNX export (#881) 
* Simplify ONNX export

* Fix ONNX CI tests
 2023-02-07 15:01:59 +08:00
marcoyang1998
1f0408b103
Support Transformer LM (#750) 
* support transformer LM

* show number of parameters during training

* update docstring

* testing files for ppl calculation

* add lm wrampper for rnn and transformer LM

* apply lm wrapper in lm shallow fusion

* small updates

* update decode.py to support LM fusion and LODR

* add export.py

* update CI and workflow

* update decoding results

* fix CI

* remove transformer LM from CI test
 2022-12-29 10:53:36 +08:00
Fangjun Kuang
88b7895adf
fix librispeech.py in multi-dataset setup (#791) 2022-12-27 13:59:55 +08:00
Daniil
b293db4baf
Tedlium3 conformer ctc2 (#696) 
* modify preparation

* small refacor

* add tedlium3 conformer_ctc2

* modify decode

* filter unk in decode

* add scaling converter

* address comments

* fix lambda function lhotse

* add implicit manifest shuffle

* refactor ctc_greedy_search

* import model arguments from train.py

* style fix

* fix ci test and last style issues

* update RESULTS

* fix RESULTS numbers

* fix label smoothing loss

* update model parameters number in RESULTS
 2022-12-13 16:13:26 +08:00
Fangjun Kuang
6533f359c9
Fix CI (#726) 
* Fix CI

* Disable shuffle for yesno.

See https://github.com/k2-fsa/icefall/issues/197
 2022-12-02 10:53:06 +08:00
marcoyang1998
4b5bc480e8
Add low-order density ratio in RNNLM shallow fusion (#678) 
* Support LODR in RNNLM shallow fusion

* fix style

* fix code style

* update workflow and CI

* update results

* propagate changes to stateless3

* add decoding results for stateless3+giga

* fix CI
 2022-11-30 17:26:05 +08:00
huangruizhe
6693d907d3
shuffle full Librispeech data (#574) 
* shuffled full/partial librispeech data

* fixed the code style issue

* Shuffled full librispeech data off-line

* Fixed style, addressed comments, and removed redandunt codes

* Used the suggested version of black

* Propagated the changes to other folders for librispeech (except
conformer_mmi and streaming_conformer_ctc)
 2022-11-27 11:26:09 +08:00
Desh Raj
d31db01037 manual correction of black formatting 2022-11-17 14:18:05 -05:00
Desh Raj
107df3b115 apply black on all files 2022-11-17 09:42:17 -05:00
Fangjun Kuang
60317120ca
Revert "Apply new Black style changes" 2022-11-17 20:19:32 +08:00
Desh Raj
d110b04ad3 apply new black formatting to all files 2022-11-16 13:06:43 -05:00
Tiance Wang
952a7b3fcc
Fix typo (#681) 
* Update add_alignment_librispeech.py

* Update scaling_converter.py
 2022-11-15 10:45:48 +08:00
Fangjun Kuang
e334e570d8
Filter utterances with number_tokens > number_feature_frames. (#604) 2022-11-12 07:57:58 +08:00
Zengwei Yao
3600ce1b5f
Apply delay penalty on transducer (#654) 
* add delay penalty

* fix CI

* fix CI
 2022-11-04 16:10:09 +08:00
Wei Kang
64aed2cdeb
Fix LG log file name (#657) 2022-11-03 23:12:35 +08:00
Wei Kang
163d929601
Add fast_beam_search_LG (#622) 
* Add fast_beam_search_LG

* add fast_beam_search_LG to commonly used recipes

* fix ci

* fix ci

* Fix error
 2022-11-03 16:29:30 +08:00
Wei Kang
d389524d45
remove tail padding for non-streaming models (#625) 2022-11-01 11:09:56 +08:00
Fangjun Kuang
7f1c0e07b6
Remove onnx and onnxruntime from requirements.txt (#640) 
* Remove onnx and onnxruntime from requirements.txt
 2022-10-31 13:44:40 +08:00
Wei Kang
581d0361cc
Fix type hints for decode.py (#638) 
* Fix type hints for decode.py

* Fix flake8
 2022-10-30 16:35:30 +08:00
Nagendra Goel
6709bf1e63
Update train.py (#635) 
Add the missing step to add the arguments to the parser.
 2022-10-28 10:23:32 +08:00
Fangjun Kuang
d69bb826ed
Support exporting LSTM with projection to ONNX (#621) 
* Support exporting LSTM with projection to ONNX

* Add missing files

* small fixes
 2022-10-18 11:25:31 +08:00
Fangjun Kuang
d1f16a04bd
fix type hints for decode.py (#623) 2022-10-18 06:56:12 +08:00
Fangjun Kuang
a66e74b92f
Fix links in the doc (#619) 2022-10-14 12:23:47 +08:00
Fangjun Kuang
c39cba5191
Support exporting to ONNX for the wenetspeech recipe (#615) 
* Support exporting to ONNX for the wenetspeech recipe
 2022-10-13 15:17:20 +08:00
Zengwei Yao
aa58c2ee02
Modify ActivationBalancer for speed (#612) 
* add a probability to apply ActivationBalancer

* minor fix

* minor fix
 2022-10-13 15:14:28 +08:00
Fangjun Kuang
1c07d2fb37
Remove all-in-one for onnx export (#614) 
* Remove all-in-one for onnx export

* Exit on error for CI
 2022-10-12 10:34:06 +08:00