Daniel Povey
|
244633660d
|
Implement ConvNorm2d and use it in frontend after convnext
|
2022-12-20 20:28:03 +08:00 |
|
Daniel Povey
|
71880409cc
|
Bug fix; also make the final norm of Conv2dSubsampling a ConvNorm1d
|
2022-12-20 19:44:04 +08:00 |
|
Daniel Povey
|
3b4b33af58
|
Avoid infinities in padding frames
|
2022-12-20 19:19:45 +08:00 |
|
Daniel Povey
|
494139d27a
|
Replace BasicNorm of encoder layers with ConvNorm1d
|
2022-12-20 19:15:14 +08:00 |
|
Daniel Povey
|
f59697555f
|
Add BasicNorm on output of Conv2dSubsampling module
|
2022-12-20 15:00:01 +08:00 |
|
Daniel Povey
|
d2b272ab50
|
Add back 2 conformer layers in 1st stack.
|
2022-12-20 13:54:06 +08:00 |
|
Daniel Povey
|
5fa8de5c05
|
Implement layerdrop per-sequence for convnext; lower, slower-decreasing layerdrop rate.
|
2022-12-20 13:51:08 +08:00 |
|
Daniel Povey
|
28cac1c2dc
|
Merge debugging changes to optimizer.
|
2022-12-20 13:01:50 +08:00 |
|
Daniel Povey
|
b546ac866c
|
Merge change from 726, set batch count at start of loop for repeatability.
|
2022-12-20 11:48:50 +08:00 |
|
Daniel Povey
|
5c11e92d4a
|
Adjust warmup duration of layerdrop_prob
|
2022-12-20 00:12:10 +08:00 |
|
Daniel Povey
|
473bb338d6
|
Merge branch 'scaled_adam_exp734' into scaled_adam_exp738
|
2022-12-20 00:10:19 +08:00 |
|
Daniel Povey
|
2cc5bc18be
|
Merge branch 'scaled_adam_exp731' into scaled_adam_exp737
# Conflicts:
# egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
|
2022-12-20 00:04:49 +08:00 |
|
Daniel Povey
|
6277a5ab4b
|
Merge branch 'scaled_adam_exp725' into scaled_adam_exp736
# Conflicts:
# egs/librispeech/ASR/pruned_transducer_stateless7/scaling.py
# egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
|
2022-12-20 00:01:38 +08:00 |
|
Daniel Povey
|
c7d15dacc6
|
Revert "Increase layerdrop_prob of ConvNeXt, and make it warm up faster."
This reverts commit 111a0aa3c73618eee7986291780383f166aac85d.
|
2022-12-18 14:31:59 +08:00 |
|
Daniel Povey
|
f439399ced
|
Adjust batch count w.r.t. reference duration
|
2022-12-18 14:25:23 +08:00 |
|
Daniel Povey
|
5e1bf8b8ec
|
Add BasicNorm to ConvNeXt; increase prob given to CutoffEstimator; adjust default probs of ActivationBalancer.
|
2022-12-18 14:14:15 +08:00 |
|
Daniel Povey
|
0341ff1ec5
|
One more convnext layer, two fewer conformer layers.
|
2022-12-17 22:00:58 +08:00 |
|
Daniel Povey
|
a424a73881
|
Increase ratio of convnext from 3 to 4.
|
2022-12-17 21:58:59 +08:00 |
|
Daniel Povey
|
9a72567b7f
|
Restore two nonlinearities.
|
2022-12-17 21:57:21 +08:00 |
|
Daniel Povey
|
598f52cbac
|
Remove 2 ConvNeXt layers.
|
2022-12-17 21:53:46 +08:00 |
|
Daniel Povey
|
cc739b193a
|
Implement memory-saving measure in randomized modules
|
2022-12-17 18:21:00 +08:00 |
|
Daniel Povey
|
111a0aa3c7
|
Increase layerdrop_prob of ConvNeXt, and make it warm up faster.
|
2022-12-17 16:44:57 +08:00 |
|
Daniel Povey
|
286b2021c2
|
Convert batch index to int
|
2022-12-17 16:31:45 +08:00 |
|
Daniel Povey
|
2c0cec86a3
|
Set batch count less frequently
|
2022-12-17 16:31:24 +08:00 |
|
Daniel Povey
|
dfeafd6aa8
|
Remove print statement in CutoffEstimator
|
2022-12-17 16:28:45 +08:00 |
|
Daniel Povey
|
29df07ba2c
|
Add memory cutoff on ActivationBalancer and Whiten
|
2022-12-17 16:20:15 +08:00 |
|
Daniel Povey
|
96daf7a00f
|
Bug fix; remove BasicNorm; add one more ConvNeXt layer.
|
2022-12-17 16:11:54 +08:00 |
|
Daniel Povey
|
744dca1c9b
|
Merge branch 'scaled_adam_exp724' into scaled_adam_exp726
|
2022-12-17 15:46:57 +08:00 |
|
Daniel Povey
|
86bb0623e9
|
Remove query from AttentionDownsample, rename to SimpleDownsample
|
2022-12-17 13:45:30 +08:00 |
|
Daniel Povey
|
ed7e01448c
|
Remove query in AttentionDownsample, rename to SimpleDownsample.
|
2022-12-17 13:44:08 +08:00 |
|
Daniel Povey
|
35b63c1387
|
Revert "Reduce const_attention_rate"
This reverts commit bc002a9eda8ac912cff235460dcdef2fd51b2f19.
|
2022-12-17 13:27:17 +08:00 |
|
Daniel Povey
|
b9326e1ef2
|
Fix to print statement
|
2022-12-16 18:07:43 +08:00 |
|
Daniel Povey
|
4eb3e97848
|
Remove bias from SimpleUpsample, add one to AttentionDownsample
|
2022-12-16 17:59:15 +08:00 |
|
Daniel Povey
|
bc002a9eda
|
Reduce const_attention_rate
|
2022-12-16 16:30:49 +08:00 |
|
Daniel Povey
|
66465c8be4
|
Give attention_skip_rate a longer tail
|
2022-12-16 15:12:04 +08:00 |
|
Daniel Povey
|
8e6c7ef3e2
|
Adjust default prob of ActivationBalancer.
|
2022-12-16 15:08:46 +08:00 |
|
Daniel Povey
|
56ac7354df
|
Remove LinearWithAuxLoss; simplify schedule of prob in ActivationBalancer.
|
2022-12-16 15:07:42 +08:00 |
|
Daniel Povey
|
3213c18a22
|
Changes to schedules: _whitening_schedule longer, min_abs schedule on attention_squeeze+nonlin_attention shorter; dip in conv_skip_rate.
|
2022-12-16 14:58:15 +08:00 |
|
Daniel Povey
|
e84f525840
|
Fix test condition
|
2022-12-16 12:24:54 +08:00 |
|
Daniel Povey
|
53ab18a862
|
Ditch caching_eval; reduce params more.
|
2022-12-16 00:22:44 +08:00 |
|
Daniel Povey
|
083e5474c4
|
Reduce ConvNeXt parameters.
|
2022-12-16 00:21:04 +08:00 |
|
Daniel Povey
|
8d9301e225
|
Remove potentially wrong typing info
|
2022-12-15 23:47:41 +08:00 |
|
Daniel Povey
|
6caaa4e9c6
|
Bug fix in caching_eval, may make no difference.
|
2022-12-15 23:32:29 +08:00 |
|
Daniel Povey
|
f5d4fb092d
|
Bug fix in caching_eval
|
2022-12-15 23:24:36 +08:00 |
|
Daniel Povey
|
d26ee2bf81
|
Try to implement caching evaluation for memory efficient training
|
2022-12-15 23:06:40 +08:00 |
|
Daniel Povey
|
f66c1600f4
|
Bug fix to printing code
|
2022-12-15 21:55:23 +08:00 |
|
Daniel Povey
|
076b18db60
|
Implement Nextformer-style frontend
|
2022-12-15 21:48:32 +08:00 |
|
Daniel Povey
|
864ff96322
|
Remove nonlin_skip_rate, introduce conv_skip_rate.
|
2022-12-15 19:27:29 +08:00 |
|
Daniel Povey
|
1506b83c7b
|
Change nonlin_skip_rate to be conv_skip_rate.
|
2022-12-15 19:25:21 +08:00 |
|
Daniel Povey
|
37a8c30136
|
Merge branch 'scaled_adam_exp699' into scaled_adam_exp711
|
2022-12-15 00:24:56 +08:00 |
|