mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-09-19 22:14:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,874 Commits 5 Branches 3 Tags
Commit Graph

1874 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Povey
049174722f Change BasicNorm by adding 1+eps denominator; fix to (unused) DoubleSwish, revert to old status. 2022-12-23 13:16:51 +08:00
Daniel Povey
cff350d8de Merge branch 'scaled_adam_exp760' into scaled_adam_exp765 2022-12-23 13:08:09 +08:00
Daniel Povey
edd6e0faf1 Add whitening on ConvNeXt module outputs; change grad_scale on whiten of Conv2dSubsampling. 2022-12-23 11:35:11 +08:00
Daniel Povey
ade7db54e3 Revert BasicNorm to its previous status, without the bias 2022-12-22 23:47:21 +08:00
Daniel Povey
b2125535fb Remove mistaken factor of 4.0 2022-12-22 23:19:16 +08:00
Daniel Povey
49bf3ddc66 Add whitening module at end of Conv2dSubsampling layer 2022-12-22 23:14:30 +08:00
Daniel Povey
2e6610af5e Fix diagnostics.py re backoff for eigs 2022-12-22 23:14:28 +08:00
Daniel Povey
e5b047a814 Merge branch 'scaled_adam_exp759' into scaled_adam_exp760 2022-12-22 17:38:17 +08:00
Daniel Povey
56fcb14e18 Merge branch 'scaled_adam_exp758' into scaled_adam_exp759 
# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless7/scaling.py
 2022-12-22 17:37:22 +08:00
Daniel Povey
1dbe1e4086 Bug fix regarding bias 2022-12-22 17:35:29 +08:00
Daniel Povey
180c440e63 Make BasicNorm after convnext1 operate over all frequency bins. 2022-12-22 17:25:30 +08:00
Daniel Povey
dd7257f01b Replace 1st ConvNorm2d with BasicNorm, remove the 2nd. 2022-12-22 16:50:52 +08:00
Daniel Povey
a0b2276f68 Subtract bias after scaling 2022-12-22 15:45:45 +08:00
Daniel Povey
d31e2e12c6 Change for memory efficiency 2022-12-22 15:20:58 +08:00
Daniel Povey
903955f5d9 Add bias to BasicNorm 2022-12-22 15:14:49 +08:00
Daniel Povey
b39cde85c8 Implement bias in BasicNorm 2022-12-22 14:59:29 +08:00
Daniel Povey
5aa874d8e3 Change layerdrop schedule of convnext, now ends at 0 2022-12-21 23:58:13 +08:00
Daniel Povey
678be7a2eb Revert ConvNorm1d to BasicNorm in Conv2dSubsampling and ZipformerLayer to BasicNorm 2022-12-21 23:53:13 +08:00
Daniel Povey
0995970f29 Decrease hidden_ratio of ConvNeXt from 4 to 3. 2022-12-21 18:43:11 +08:00
Daniel Povey
39e7c613c7 Add balancer to ConvNeXt 2022-12-21 18:41:05 +08:00
Daniel Povey
11f68afa1f Revert "Remove memory-cutoff from ActivationBalancer." 
This reverts commit 5afe0e78556e2e76750cae64008c9dd5c1931c5c.
 2022-12-21 18:39:16 +08:00
Daniel Povey
829e4bd4db Bug fix in save-bad-model code 2022-12-21 15:33:58 +08:00
Daniel Povey
788c4d97f1 Remove memory-cutoff from ActivationBalancer. 2022-12-21 15:09:26 +08:00
Daniel Povey
266e71cc79 Save checkpoint on failure. 2022-12-21 15:09:16 +08:00
Daniel Povey
96d167a2ec Reduce floor on conv_min 2022-12-21 15:08:59 +08:00
Daniel Povey
05bcfd3b07 Make Whiten module update its prob every time 2022-12-21 12:56:37 +08:00
Daniel Povey
c097c13720 Change memory cutoff for ActivationBalancer; remove it for Whiten 2022-12-21 11:25:17 +08:00
Daniel Povey
4d61d39d36 Merge branch 'scaled_adam_exp747' into scaled_adam_exp748 2022-12-20 23:23:49 +08:00
Daniel Povey
3ef2a1d81e Make some of the layer-skipping logic be done per sequence. 2022-12-20 22:26:30 +08:00
Daniel Povey
244633660d Implement ConvNorm2d and use it in frontend after convnext 2022-12-20 20:28:03 +08:00
Daniel Povey
71880409cc Bug fix; also make the final norm of Conv2dSubsampling a ConvNorm1d 2022-12-20 19:44:04 +08:00
Daniel Povey
3b4b33af58 Avoid infinities in padding frames 2022-12-20 19:19:45 +08:00
Daniel Povey
494139d27a Replace BasicNorm of encoder layers with ConvNorm1d 2022-12-20 19:15:14 +08:00
Daniel Povey
f59697555f Add BasicNorm on output of Conv2dSubsampling module 2022-12-20 15:00:01 +08:00
Daniel Povey
d2b272ab50 Add back 2 conformer layers in 1st stack. 2022-12-20 13:54:06 +08:00
Daniel Povey
5fa8de5c05 Implement layerdrop per-sequence for convnext; lower, slower-decreasing layerdrop rate. 2022-12-20 13:51:08 +08:00
Daniel Povey
28cac1c2dc Merge debugging changes to optimizer. 2022-12-20 13:01:50 +08:00
Daniel Povey
b546ac866c Merge change from 726, set batch count at start of loop for repeatability. 2022-12-20 11:48:50 +08:00
Daniel Povey
5c11e92d4a Adjust warmup duration of layerdrop_prob 2022-12-20 00:12:10 +08:00
Daniel Povey
473bb338d6 Merge branch 'scaled_adam_exp734' into scaled_adam_exp738 2022-12-20 00:10:19 +08:00
Daniel Povey
2cc5bc18be Merge branch 'scaled_adam_exp731' into scaled_adam_exp737 
# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
 2022-12-20 00:04:49 +08:00
Daniel Povey
6277a5ab4b Merge branch 'scaled_adam_exp725' into scaled_adam_exp736 
# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless7/scaling.py
#	egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
 2022-12-20 00:01:38 +08:00
Daniel Povey
c7d15dacc6 Revert "Increase layerdrop_prob of ConvNeXt, and make it warm up faster." 
This reverts commit 111a0aa3c73618eee7986291780383f166aac85d.
 2022-12-18 14:31:59 +08:00
Daniel Povey
f439399ced Adjust batch count w.r.t. reference duration 2022-12-18 14:25:23 +08:00
Daniel Povey
5e1bf8b8ec Add BasicNorm to ConvNeXt; increase prob given to CutoffEstimator; adjust default probs of ActivationBalancer. 2022-12-18 14:14:15 +08:00
Daniel Povey
0341ff1ec5 One more convnext layer, two fewer conformer layers. 2022-12-17 22:00:58 +08:00
Daniel Povey
a424a73881 Increase ratio of convnext from 3 to 4. 2022-12-17 21:58:59 +08:00
Daniel Povey
9a72567b7f Restore two nonlinearities. 2022-12-17 21:57:21 +08:00
Daniel Povey
598f52cbac Remove 2 ConvNeXt layers. 2022-12-17 21:53:46 +08:00
Daniel Povey
cc739b193a Implement memory-saving measure in randomized modules 2022-12-17 18:21:00 +08:00