mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,665 Commits 5 Branches 3 Tags
Commit Graph

334 Commits

Author SHA1 Message Date
Daniel Povey
460fb945ec Merge branch 'scaled_adam_exp813' into scaled_adam_exp820 2023-01-01 22:12:10 +08:00
Daniel Povey
977d412690 Cosmetic fix 2023-01-01 21:43:14 +08:00
Daniel Povey
60d491eee6 Bug fix 2023-01-01 14:31:28 +08:00
Daniel Povey
287bd120be Reduce min_abs of zipformer balancer1; constraints on eps of Conv2dSubsampling.out_norm 2023-01-01 14:28:18 +08:00
Daniel Povey
a2815ea0df Increase max_abs of ZipformerEncoderLayer.balancer2 from 1.0 to 4.0. 2023-01-01 00:00:26 +08:00
Daniel Povey
63472a19b1 Whitespace fix 2022-12-31 23:50:09 +08:00
Daniel Povey
4a4d12c994 Revert kernel size of convnext2 from 5x5 to 7x7 2022-12-31 21:52:11 +08:00
Daniel Povey
d0ae60400e Decrease convnext1 kernel size from 7x7 to 5x5 2022-12-31 17:19:02 +08:00
Daniel Povey
d48b2ccb45 Reduce kernel size of convnext2 from 7 to 5. 2022-12-31 17:10:31 +08:00
Daniel Povey
c533c30442 Increase final conv_skip_rate from 0.0 to 0.01 2022-12-31 15:10:52 +08:00
Daniel Povey
577c3ad390 Adjust balancers of modules; most significant change is to make min_abs of ff2 balancer from 0.5 to 0.1 2022-12-31 14:38:00 +08:00
Daniel Povey
c15578d0bb Add balancer_ff2 to avoid too small ff2 module 2022-12-31 01:09:17 +08:00
Daniel Povey
9ee4472f36 Decrease min_abs at end of feedforward modules from 0.5 to 0.1. 2022-12-30 23:29:03 +08:00
Daniel Povey
da0623aa7f Add another balancer to ZipformerEncoderLayer, prior to output. 2022-12-30 14:35:49 +08:00
Daniel Povey
59be36181c Replace ActivationBalancer with Balancer 2022-12-29 20:34:46 +08:00
Daniel Povey
c6bad1ee4f Start ff modules with larger initial_scale 2022-12-29 18:50:12 +08:00
Daniel Povey
fbdb12cf77 Remove ZipformerEncoder.norm 2022-12-29 16:00:34 +08:00
Daniel Povey
0de1184c6d Fix min_abs for AttentionSqueeze 2022-12-29 15:24:13 +08:00
Daniel Povey
03e1f7dc01 Multiply min_abs values in line of encoder residuals by 4. 2022-12-29 12:49:04 +08:00
Daniel Povey
71d7843654 Re-introduce bias into BasicNorm and replace eps with log_scale. 2022-12-26 21:22:00 +08:00
Daniel Povey
920ed685ac Change how bypass_scale works, src = src * bypass_scale + src_orig * (1.0 - bypass_scale) 2022-12-26 14:27:16 +08:00
Daniel Povey
11f5454b6a Increase eps_max in norm_final from 3 to 4. 2022-12-26 13:47:23 +08:00
Daniel Povey
3d6ee443e3 Revert some recent changes that may not have been helpful. 2022-12-24 21:17:43 +08:00
Daniel Povey
43f2a8d50b Make norm_final apply to delta, not src 2022-12-24 18:44:42 +08:00
Daniel Povey
72420eef10 Change final layerdrop_rate from 0.0 to 0.015. 2022-12-23 21:56:13 +08:00
Daniel Povey
2e0f4de8ff Apply limit on BasicNorm.eps more effectively using limit_param_value; add final norm to Zipformer. 2022-12-23 15:59:51 +08:00
Daniel Povey
cff350d8de Merge branch 'scaled_adam_exp760' into scaled_adam_exp765 2022-12-23 13:08:09 +08:00
Daniel Povey
edd6e0faf1 Add whitening on ConvNeXt module outputs; change grad_scale on whiten of Conv2dSubsampling. 2022-12-23 11:35:11 +08:00
Daniel Povey
ade7db54e3 Revert BasicNorm to its previous status, without the bias 2022-12-22 23:47:21 +08:00
Daniel Povey
b2125535fb Remove mistaken factor of 4.0 2022-12-22 23:19:16 +08:00
Daniel Povey
49bf3ddc66 Add whitening module at end of Conv2dSubsampling layer 2022-12-22 23:14:30 +08:00
Daniel Povey
180c440e63 Make BasicNorm after convnext1 operate over all frequency bins. 2022-12-22 17:25:30 +08:00
Daniel Povey
dd7257f01b Replace 1st ConvNorm2d with BasicNorm, remove the 2nd. 2022-12-22 16:50:52 +08:00
Daniel Povey
d31e2e12c6 Change for memory efficiency 2022-12-22 15:20:58 +08:00
Daniel Povey
5aa874d8e3 Change layerdrop schedule of convnext, now ends at 0 2022-12-21 23:58:13 +08:00
Daniel Povey
678be7a2eb Revert ConvNorm1d to BasicNorm in Conv2dSubsampling and ZipformerLayer to BasicNorm 2022-12-21 23:53:13 +08:00
Daniel Povey
0995970f29 Decrease hidden_ratio of ConvNeXt from 4 to 3. 2022-12-21 18:43:11 +08:00
Daniel Povey
39e7c613c7 Add balancer to ConvNeXt 2022-12-21 18:41:05 +08:00
Daniel Povey
4d61d39d36 Merge branch 'scaled_adam_exp747' into scaled_adam_exp748 2022-12-20 23:23:49 +08:00
Daniel Povey
3ef2a1d81e Make some of the layer-skipping logic be done per sequence. 2022-12-20 22:26:30 +08:00
Daniel Povey
244633660d Implement ConvNorm2d and use it in frontend after convnext 2022-12-20 20:28:03 +08:00
Daniel Povey
71880409cc Bug fix; also make the final norm of Conv2dSubsampling a ConvNorm1d 2022-12-20 19:44:04 +08:00
Daniel Povey
494139d27a Replace BasicNorm of encoder layers with ConvNorm1d 2022-12-20 19:15:14 +08:00
Daniel Povey
f59697555f Add BasicNorm on output of Conv2dSubsampling module 2022-12-20 15:00:01 +08:00
Daniel Povey
5fa8de5c05 Implement layerdrop per-sequence for convnext; lower, slower-decreasing layerdrop rate. 2022-12-20 13:51:08 +08:00
Daniel Povey
5c11e92d4a Adjust warmup duration of layerdrop_prob 2022-12-20 00:12:10 +08:00
Daniel Povey
473bb338d6 Merge branch 'scaled_adam_exp734' into scaled_adam_exp738 2022-12-20 00:10:19 +08:00
Daniel Povey
2cc5bc18be Merge branch 'scaled_adam_exp731' into scaled_adam_exp737 
# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
 2022-12-20 00:04:49 +08:00
Daniel Povey
6277a5ab4b Merge branch 'scaled_adam_exp725' into scaled_adam_exp736 
# Conflicts:
#	egs/librispeech/ASR/pruned_transducer_stateless7/scaling.py
#	egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
 2022-12-20 00:01:38 +08:00
Daniel Povey
c7d15dacc6 Revert "Increase layerdrop_prob of ConvNeXt, and make it warm up faster." 
This reverts commit 111a0aa3c73618eee7986291780383f166aac85d.
 2022-12-18 14:31:59 +08:00