mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,225 Commits 5 Branches 3 Tags
Commit Graph

83 Commits

Author SHA1 Message Date
Fangjun Kuang
8905ef5d59 Fix gigaspeech dataset iterator. 
Previously, it was reset after every epoch, which may cause it to
always use the first part of the gigaspeech dataset if you choose
a small --giga-prob.
 2025-11-28 10:51:21 +08:00
Fangjun Kuang
fba5e67d5e
Fix CI tests. (#1974) 
- Introduce unified AMP helpers (create_grad_scaler, torch_autocast) to handle 
  deprecations in PyTorch ≥2.3.0

- Replace direct uses of torch.cuda.amp.GradScaler and torch.cuda.amp.autocast 
  with the new utilities across all training and inference scripts

- Update all torch.load calls to include weights_only=False for compatibility with 
  newer PyTorch versions
 2025-07-01 13:47:55 +08:00
Fangjun Kuang
d4d4f281ec
Revert "Replace deprecated pytorch methods (#1814)" (#1841) 
This reverts commit 3e4da5f78160d3dba3bdf97968bd7ceb8c11631f.
 2024-12-18 16:49:57 +08:00
Li Peng
3e4da5f781
Replace deprecated pytorch methods (#1814) 
* Replace deprecated pytorch methods

- torch.cuda.amp.GradScaler(...) => torch.amp.GradScaler("cuda", ...)
- torch.cuda.amp.autocast(...) => torch.amp.autocast("cuda", ...)

* Replace `with autocast(...)` with `with autocast("cuda", ...)`


Co-authored-by: Li Peng <lipeng@unisound.ai>
 2024-12-16 10:24:16 +08:00
zr_jin
242002e0bd
Strengthened style constraints (#1527) 2024-03-04 23:28:04 +08:00
Yifan Yang
5dfc3ed7f9
Fix buffer size of DynamicBucketingSampler (#1468) 
* Fix buffer size

* Fix for flake8

---------

Co-authored-by: yifanyeung <yifanyeung@yifanyeung.local>
 2024-01-21 02:10:42 +08:00
Karel Vesely
716b82cc3a
streaming_decode.py, relax the audio range from [-1,+1] to [-10,+10] (#1448) 
- some AudioTransform classes produce audio signals out of range [-1,+1]
   - Resample produced 1.0079
   - The range [-10,+10] was chosen to still be able to reliably
     distinguish from the [-32k,+32k] signal...
- this is related to : https://github.com/lhotse-speech/lhotse/issues/1254
 2024-01-05 10:21:27 +08:00
Fangjun Kuang
8136ad775b
Use high_freq -400 in computing fbank features. (#1447) 
See also https://github.com/k2-fsa/sherpa-onnx/issues/514
 2024-01-04 13:59:32 +08:00
zr_jin
d2bd0933b1
Compatibility with the latest Lhotse (#1314) 2023-10-17 21:22:32 +08:00
Fangjun Kuang
34e40a86b3
Fix exporting decoder model to onnx (#1264) 
* Use torch.jit.script() to export the decoder model

See also https://github.com/k2-fsa/sherpa-onnx/issues/327
 2023-09-22 09:57:15 +08:00
Fangjun Kuang
f5dc957d44
Fix CI tests (#1266) 2023-09-21 21:16:14 +08:00
zr_jin
a81396b482
Use tokens.txt to replace bpe.model (#1162) 2023-08-12 16:53:59 +08:00
Fangjun Kuang
1dbbd7759e
Add tests for subsample.py and fix typos (#1180) 2023-07-25 14:46:18 +08:00
Yifan Yang
ca60ced213
Fix typo (#1114) 
* Fix typo for zipformer

* Fix typo for pruned_transducer_stateless7

* Fix typo for pruned_transducer_stateless7_ctc

* Fix typo for pruned_transducer_stateless7_ctc_bs

* Fix typo for pruned_transducer_stateless7_streaming

* Fix typo for pruned_transducer_stateless7_streaming_multi

* Fix file permissions for pruned_transducer_stateless7_streaming_multi

* Fix typo for pruned_transducer_stateless8

* Fix typo for pruned_transducer_stateless6

* Fix typo for pruned_transducer_stateless5

* Fix typo for pruned_transducer_stateless4

* Fix typo for pruned_transducer_stateless3
 2023-06-02 14:12:42 +08:00
Yifan Yang
d67a49afe4
Add multidataset (#1010) 
* Add Common Voice for multidataset

* Add prepare_multidataset.sh

* Add dataset mixing


* Update prepare_multidataset.sh

* Update prepare_giga_speech.sh

* update comments

* Add split and shuffle mechanism

* Add multi-dataset train

* Fix for deleting

* Fix for modifying

* Add comments

* Change type for perturb_speed

* Fix for style check

* Small fix

* Add filter

* Remove warning
 2023-04-21 18:09:41 +08:00
Zengwei Yao
5f066d3d53
support decoding and computing RTF on test sets with onnx models (#995) 
* support decode and compute RTF on test sets with onnx models

* support onnx export and decode in pruned_transducer_stateless
 2023-04-12 19:04:50 +08:00
Zengwei Yao
136aa94d57
remove duplicated lines (#988) 2023-04-06 17:47:33 +08:00
marcoyang1998
d337398d29
Shallow fusion for Aishell (#954) 
* add shallow fusion and LODR for aishell

* update RESULTS

* add save by iterations
 2023-04-03 16:20:29 +08:00
Fangjun Kuang
a632b24c35
Export int8 quantized models for non-streaming Zipformer. (#977) 
* Export int8 quantized models for non-streaming Zipformer.

* Delete export-onnx.py

* Export int8 models for other folders
 2023-03-31 22:46:19 +08:00
Zengwei Yao
2a5a75cb56
add option of using full attention for streaming model decoding (#975) 2023-03-30 14:30:13 +08:00
Fangjun Kuang
8c3ea93fc8
Save meta data to exported ONNX models (#968) 2023-03-27 11:39:29 +08:00
Fangjun Kuang
f5de2e90c6
Fix style issues. (#937) 2023-03-08 22:56:04 +08:00
pehonnet
07243d136a
remove key from result filename (#936) 
Co-authored-by: pe-honnet <pe.honnet@telepathy.ai>
 2023-03-08 21:06:07 +08:00
Fangjun Kuang
2b995639b7
Add ONNX support for Zipformer and ConvEmformer (#884) 2023-02-09 00:02:38 +08:00
Fangjun Kuang
7ae03f6c88
Add onnx export support for pruned_transducer_stateless5 (#883) 2023-02-07 17:47:08 +08:00
Fangjun Kuang
8d3810e289
Simplify ONNX export (#881) 
* Simplify ONNX export

* Fix ONNX CI tests
 2023-02-07 15:01:59 +08:00
marcoyang1998
1f0408b103
Support Transformer LM (#750) 
* support transformer LM

* show number of parameters during training

* update docstring

* testing files for ppl calculation

* add lm wrampper for rnn and transformer LM

* apply lm wrapper in lm shallow fusion

* small updates

* update decode.py to support LM fusion and LODR

* add export.py

* update CI and workflow

* update decoding results

* fix CI

* remove transformer LM from CI test
 2022-12-29 10:53:36 +08:00
Fangjun Kuang
88b7895adf
fix librispeech.py in multi-dataset setup (#791) 2022-12-27 13:59:55 +08:00
Daniil
b293db4baf
Tedlium3 conformer ctc2 (#696) 
* modify preparation

* small refacor

* add tedlium3 conformer_ctc2

* modify decode

* filter unk in decode

* add scaling converter

* address comments

* fix lambda function lhotse

* add implicit manifest shuffle

* refactor ctc_greedy_search

* import model arguments from train.py

* style fix

* fix ci test and last style issues

* update RESULTS

* fix RESULTS numbers

* fix label smoothing loss

* update model parameters number in RESULTS
 2022-12-13 16:13:26 +08:00
Fangjun Kuang
6533f359c9
Fix CI (#726) 
* Fix CI

* Disable shuffle for yesno.

See https://github.com/k2-fsa/icefall/issues/197
 2022-12-02 10:53:06 +08:00
marcoyang1998
4b5bc480e8
Add low-order density ratio in RNNLM shallow fusion (#678) 
* Support LODR in RNNLM shallow fusion

* fix style

* fix code style

* update workflow and CI

* update results

* propagate changes to stateless3

* add decoding results for stateless3+giga

* fix CI
 2022-11-30 17:26:05 +08:00
huangruizhe
6693d907d3
shuffle full Librispeech data (#574) 
* shuffled full/partial librispeech data

* fixed the code style issue

* Shuffled full librispeech data off-line

* Fixed style, addressed comments, and removed redandunt codes

* Used the suggested version of black

* Propagated the changes to other folders for librispeech (except
conformer_mmi and streaming_conformer_ctc)
 2022-11-27 11:26:09 +08:00
Desh Raj
d31db01037 manual correction of black formatting 2022-11-17 14:18:05 -05:00
Desh Raj
107df3b115 apply black on all files 2022-11-17 09:42:17 -05:00
Fangjun Kuang
60317120ca
Revert "Apply new Black style changes" 2022-11-17 20:19:32 +08:00
Desh Raj
d110b04ad3 apply new black formatting to all files 2022-11-16 13:06:43 -05:00
Tiance Wang
952a7b3fcc
Fix typo (#681) 
* Update add_alignment_librispeech.py

* Update scaling_converter.py
 2022-11-15 10:45:48 +08:00
Fangjun Kuang
e334e570d8
Filter utterances with number_tokens > number_feature_frames. (#604) 2022-11-12 07:57:58 +08:00
Zengwei Yao
3600ce1b5f
Apply delay penalty on transducer (#654) 
* add delay penalty

* fix CI

* fix CI
 2022-11-04 16:10:09 +08:00
Wei Kang
64aed2cdeb
Fix LG log file name (#657) 2022-11-03 23:12:35 +08:00
Wei Kang
163d929601
Add fast_beam_search_LG (#622) 
* Add fast_beam_search_LG

* add fast_beam_search_LG to commonly used recipes

* fix ci

* fix ci

* Fix error
 2022-11-03 16:29:30 +08:00
Wei Kang
d389524d45
remove tail padding for non-streaming models (#625) 2022-11-01 11:09:56 +08:00
Fangjun Kuang
7f1c0e07b6
Remove onnx and onnxruntime from requirements.txt (#640) 
* Remove onnx and onnxruntime from requirements.txt
 2022-10-31 13:44:40 +08:00
Wei Kang
581d0361cc
Fix type hints for decode.py (#638) 
* Fix type hints for decode.py

* Fix flake8
 2022-10-30 16:35:30 +08:00
Nagendra Goel
6709bf1e63
Update train.py (#635) 
Add the missing step to add the arguments to the parser.
 2022-10-28 10:23:32 +08:00
Fangjun Kuang
d69bb826ed
Support exporting LSTM with projection to ONNX (#621) 
* Support exporting LSTM with projection to ONNX

* Add missing files

* small fixes
 2022-10-18 11:25:31 +08:00
Fangjun Kuang
d1f16a04bd
fix type hints for decode.py (#623) 2022-10-18 06:56:12 +08:00
Fangjun Kuang
a66e74b92f
Fix links in the doc (#619) 2022-10-14 12:23:47 +08:00
Fangjun Kuang
c39cba5191
Support exporting to ONNX for the wenetspeech recipe (#615) 
* Support exporting to ONNX for the wenetspeech recipe
 2022-10-13 15:17:20 +08:00
Zengwei Yao
aa58c2ee02
Modify ActivationBalancer for speed (#612) 
* add a probability to apply ActivationBalancer

* minor fix

* minor fix
 2022-10-13 15:14:28 +08:00