mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,376 Commits 5 Branches 3 Tags
Commit Graph

1376 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Povey
05689f6354 Add skip connections as in normal U-net 2022-10-29 19:57:12 +08:00
Daniel Povey
bba454a349 Make decoder group size equal to 4. 2022-10-29 17:08:19 +08:00
Daniel Povey
ff03ec88a5 Tuning change to num encoder layers, inspired by relative param importance. 2022-10-29 15:56:02 +08:00
Daniel Povey
f995426324 Reduce min of bypass_scale from 0.5 to 0.3, and make it not applied in test mode. 2022-10-29 15:40:47 +08:00
Daniel Povey
435d0dec71 Reduce dim of linear positional encoding in attention layers. 2022-10-29 15:31:34 +08:00
Daniel Povey
96ea4cf1be Have 6 different encoder stacks, U-shaped network. 2022-10-28 20:36:45 +08:00
Daniel Povey
7b57a34227 Have 4 encoder stacks 2022-10-28 19:32:14 +08:00
Daniel Povey
de9a6ebd6c Fix bug RE seq lengths 2022-10-28 19:26:06 +08:00
Daniel Povey
d7d5188bd9 Refactor how the downsampling is done so that it happens later, but the 1st encoder stack still operates after a subsampling of 2. 2022-10-28 19:20:21 +08:00
Daniel Povey
0a89f51dc9 Have a 3rd encoder, at downsampling factor of 8. 2022-10-28 17:42:00 +08:00
Daniel Povey
ed1b4d5e5d Refactor zipformer for more flexibility so we can change number of encoder layers. 2022-10-28 17:32:38 +08:00
Daniel Povey
e592a920b4 Merge branch 'scaled_adam_exp198b' into scaled_adam_exp202 2022-10-28 13:13:55 +08:00
Daniel Povey
a067fe8026 Fix clamping of epsilon 2022-10-28 12:50:14 +08:00
Daniel Povey
7b8a0108ea Merge branch 'scaled_adam_exp188' into scaled_adam_exp198b 2022-10-28 12:49:36 +08:00
Daniel Povey
b9f6ba1aa2 Remove some unused variables. 2022-10-28 12:01:45 +08:00
Daniel Povey
c8abba75a9 Update decode.py by copying from pruned_transducer_stateless5 and changing directory name 2022-10-28 11:19:45 +08:00
Daniel Povey
5dfa141ca5 Rename Conformer to Zipformer 2022-10-27 22:43:46 +08:00
Daniel Povey
3f05e47447 Rename conformer.py to zipformer.py 2022-10-27 22:41:48 +08:00
Daniel Povey
be5c687fbd Merging upstream/master 2022-10-27 21:04:48 +08:00
Daniel Povey
f8c531cd23 Increase bypass_scale min from 0.4 to 0.5 2022-10-27 14:59:05 +08:00
Daniel Povey
2c400115e4 Increase bypass_scale from 0.2 to 0.4. 2022-10-27 14:30:46 +08:00
Daniel Povey
a7fc6ae38c Increase floor on bypass_scale from 0.1 to 0.2. 2022-10-27 14:09:34 +08:00
Daniel Povey
938510ac9f Fix clamping of bypass scale; remove a couple unused variables. 2022-10-27 14:05:53 +08:00
Daniel Povey
bf37c7ca85 Regularize how we apply the min and max to the eps of BasicNorm 2022-10-26 12:51:20 +08:00
Daniel Povey
a0507a83a5 Change scalar_max in optim.py from 2.0 to 5.0 2022-10-25 22:58:07 +08:00
Daniel Povey
78f3cba58c Add logging about memory used. 2022-10-25 19:19:33 +08:00
Daniel Povey
6a6df19bde Hopefully make penalize_abs_values_gt more memory efficient. 2022-10-25 18:41:33 +08:00
Daniel Povey
dbfbd8016b Cast to float16 in DoubleSwish forward 2022-10-25 13:16:00 +08:00
Daniel Povey
3159b09e8f Make 20 the limit for warmup_count 2022-10-25 12:58:27 +08:00
Daniel Povey
6ebff23cb9 Reduce cutoff from 100 to 5 for estimating OOM with warmup 2022-10-25 12:53:12 +08:00
Daniel Povey
9da5526659 Changes to more accurately estimate OOM conditions 2022-10-25 12:49:18 +08:00
Daniel Povey
1e8984174b Change to warmup schedule. 2022-10-25 12:27:00 +08:00
Daniel Povey
36cb279318 More memory efficient backprop for DoubleSwish. 2022-10-25 12:21:22 +08:00
Fangjun Kuang
499ac24ecb
Install kaldifst for GitHub actions (#632) 
* Install kaldifst for GitHub actions
 2022-10-24 15:07:29 +08:00
Daniel Povey
95aaa4a8d2 Store only half precision output for softmax. 2022-10-23 21:24:46 +08:00
Daniel Povey
d3876e32c4 Make it use float16 if in amp but use clamp to avoid wrapping error 2022-10-23 21:13:23 +08:00
Daniel Povey
85657946bb Try a more exact way to round to uint8 that should prevent ever wrapping around to zero 2022-10-23 20:56:26 +08:00
Daniel Povey
d6aa386552 Fix randn to rand 2022-10-23 17:19:19 +08:00
Daniel Povey
e586cc319c Change the discretization of the sigmoid to be expectation preserving. 2022-10-23 17:11:35 +08:00
Daniel Povey
09cbc9fdab Save some memory in the autograd of DoubleSwish. 2022-10-23 16:59:43 +08:00
Daniel Povey
40588d3d8a Revert 179->180 change, i.e. change max_abs for deriv_balancer2 back from 50.0 20.0 2022-10-23 16:18:58 +08:00
Daniel Povey
5b9d166cb9 --base-lr0.075->0.5; --lr-epochs 3->3.5 2022-10-23 13:56:25 +08:00
Daniel Povey
0406d0b059 Increase max_abs in ActivationBalancer of conv module from 20 to 50 2022-10-23 13:51:51 +08:00
Daniel Povey
9e86d1f44f reduce initial scale in GradScaler 2022-10-23 00:14:38 +08:00
Daniel Povey
b7083e7aff Increase default max_factor for ActivationBalancer from 0.02 to 0.04; decrease max_abs in ConvolutionModule.deriv_balancer2 from 100.0 to 20.0 2022-10-23 00:09:21 +08:00
Daniel Povey
ad2d3c2b36 Dont print out full non-finite tensor 2022-10-22 23:03:19 +08:00
Daniel Povey
e0c1dc66da Increase probs of activation balancer and make it decay slower. 2022-10-22 22:18:38 +08:00
Daniel Povey
2964628ae1 don't do penalize_values_gt on simple_lm_proj and simple_am_proj; reduce --base-lr from 0.1 to 0.075 2022-10-22 21:12:58 +08:00
Daniel Povey
269b70122e Add hooks.py, had negleted to git add it. 2022-10-22 20:58:52 +08:00
Daniel Povey
13ffd8e823 Trying to reduce grad_scale of Whiten() from 0.02 to 0.01. 2022-10-22 20:30:05 +08:00