mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,493 Commits 5 Branches 3 Tags
Commit Graph

1493 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Povey
18ff1de337 Add debug code for attention weihts and eigs 2022-10-14 20:57:17 +08:00
Daniel Povey
96023419da Reworking of ActivationBalancer code to hopefully balance speed and effectiveness. 2022-10-14 19:20:32 +08:00
Daniel Povey
5f375be159 Merge branch 'scaled_adam_exp103b2' into scaled_adam_exp103b4 2022-10-14 15:27:10 +08:00
Daniel Povey
15b91c12d6 Reduce stats period from 10 to 4. 2022-10-14 15:14:06 +08:00
Daniel Povey
db8b9919da Reduce beta from 0.75 to 0.0. 2022-10-14 15:12:59 +08:00
Fangjun Kuang
a66e74b92f
Fix links in the doc (#619) 2022-10-14 12:23:47 +08:00
Fangjun Kuang
11bff57586
Add doc about model export (#618) 
* Add doc about model export

* fix typos
 2022-10-14 10:16:34 +08:00
Daniel Povey
ae6478c687 This should just be a cosmetic change, regularizing how we get the warmup times from the layers. 2022-10-13 19:41:28 +08:00
Fangjun Kuang
c39cba5191
Support exporting to ONNX for the wenetspeech recipe (#615) 
* Support exporting to ONNX for the wenetspeech recipe
 2022-10-13 15:17:20 +08:00
Zengwei Yao
aa58c2ee02
Modify ActivationBalancer for speed (#612) 
* add a probability to apply ActivationBalancer

* minor fix

* minor fix
 2022-10-13 15:14:28 +08:00
Daniel Povey
7d8e460a53 Revert dropout on attention scores to 0.0. 2022-10-13 15:09:50 +08:00
Daniel Povey
2a50def7c6 Simplify how the positional-embedding scores work in attention (thanks to Zengwei for this concept) 2022-10-13 15:08:00 +08:00
Daniel Povey
23d6bf7765 Fix bug when channel_dim < 0 2022-10-13 13:52:28 +08:00
Daniel Povey
b09a1b2ae6 Fix bug when channel_dim < 0 2022-10-13 13:40:43 +08:00
Daniel Povey
9270e32a51 Remove unused config value 2022-10-13 13:34:35 +08:00
Daniel Povey
63334137ee Merge branch 'scaled_adam_exp106' into scaled_adam_exp108 
# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless7/conformer.py
 2022-10-13 12:22:13 +08:00
Daniel Povey
9e30f2bf12 Make the ActivationBalancer regress to the data mean, not zero, when enforcing abs constraint. 2022-10-13 12:05:45 +08:00
Daniel Povey
49c6b6943d Change scale_factor_scale from 0.5 to 0.8 2022-10-12 20:55:52 +08:00
Daniel Povey
b736bb4840 Cosmetic improvements 2022-10-12 19:34:48 +08:00
Daniel Povey
12323025d7 Make ActivationBalancer and MaxEig more efficient. 2022-10-12 18:44:52 +08:00
Daniel Povey
eb58e6d74b Remove persistent attention scores. 2022-10-12 12:50:00 +08:00
Fangjun Kuang
1c07d2fb37
Remove all-in-one for onnx export (#614) 
* Remove all-in-one for onnx export

* Exit on error for CI
 2022-10-12 10:34:06 +08:00
Yunusemre
f3db4ea871
exporting projection layers of joiner separately for onnx (#584) 
* exporting projection layers of joiner separately for onnx
 2022-10-11 18:22:28 +08:00
KajiMaCN
0019463c83
update docs (#611) 
* update docs

Co-authored-by: unknown <mazhihao@jshcbd.cn>
Co-authored-by: KajiMaCN <moonlightshadowmzh@gmail.com>
 2022-10-11 13:24:59 +08:00
Daniel Povey
1825336841 Fix issue with diagnostics if stats is None 2022-10-11 11:05:52 +08:00
Fangjun Kuang
3614d7ff6d
Add dill to requirements.txt (#613) 
* Add dill to requirements.txt

* Disable style check for python 3.7
 2022-10-10 22:50:25 +08:00
Daniel Povey
569762397f Reduce final layerdrop_prob from 0.075 to 0.05. 2022-10-10 19:04:52 +08:00
Daniel Povey
12323f2fbf Refactor RelPosMultiheadAttention to have 2nd forward function and introduce more modules in conformer encoder layer 2022-10-10 15:27:26 +08:00
Daniel Povey
f941991331 Fix bug in choosing layers to drop 2022-10-10 13:38:36 +08:00
Daniel Povey
857b3735e7 Fix bug where fewer layers were dropped than should be; remove unnecesary print statement. 2022-10-10 13:18:40 +08:00
Daniel Povey
09c9b02f6f Increase final layerdrop prob from 0.05 to 0.075 2022-10-10 12:20:13 +08:00
Daniel Povey
9f059f7115 Fix s -> scaling for import. 2022-10-10 11:50:15 +08:00
Daniel Povey
d7f6e8eb51 Only apply ActivationBalancer with prob 0.25. 2022-10-10 00:26:31 +08:00
Daniel Povey
dece8ad204 Various fixes from debugging with nvtx, but removed the NVTX annotations. 2022-10-09 21:14:52 +08:00
Daniel Povey
bd7dce460b Reintroduce batching to the optimizer 2022-10-09 20:29:23 +08:00
Daniel Povey
00841f0f49 Remove unused code LearnedScale. 2022-10-09 16:07:31 +08:00
Daniel Povey
cf450908c6 Revert also the changes in scaled_adam_exp85 regarding warmup schedule 2022-10-09 14:26:32 +08:00
Daniel Povey
40fa33d702 Decrease initial_layerdrop_prob from 0.75 to 0.5 2022-10-09 13:59:56 +08:00
Daniel Povey
44ad73c44f For speed, drop the same num layers per job. 2022-10-09 13:40:24 +08:00
Daniel Povey
f8f200e2b2 Make layerdrop different in different processes. 2022-10-09 12:25:12 +08:00
Daniel Povey
3e137dda5b Decrease frequency of logging variance_proportion 2022-10-09 12:05:52 +08:00
Daniel Povey
e6540865f3 Do warmup by dropping out whole layers. 2022-10-09 11:50:24 +08:00
Daniel Povey
5255969544 Revert "Change warmup schedule and increase warmup_batches from 4k to 6k" 
This reverts commit 86845bd5d859ceb6f83cd83f3719c3e6641de987.
 2022-10-09 11:30:27 +08:00
Daniel Povey
d467338837 Limit bypass scale to >= 0.1 2022-10-08 21:37:21 +08:00
Daniel Povey
bc9fbe2579 Bug fix 2022-10-08 21:06:09 +08:00
Daniel Povey
9023fe7151 Change the initial keep-prob back from 0.25 to 0.5 2022-10-08 20:55:15 +08:00
Daniel Povey
97a0fbe44b Make the bypass scale trainable. 2022-10-08 20:32:49 +08:00
Daniel Povey
86845bd5d8 Change warmup schedule and increase warmup_batches from 4k to 6k 2022-10-08 19:10:26 +08:00
Daniel Povey
2631f05c1f Make it start warming up from the very start, and increase warmup_batches to 6k 2022-10-08 19:09:41 +08:00
Daniel Povey
5c99e97c3b Decrease initial keep_prob to 0.25. 2022-10-08 18:35:59 +08:00