mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-09-20 06:24:21 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,842 Commits 5 Branches 3 Tags
Commit Graph

1842 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Povey
b7876baed6 Remove dynamic weights in SimpleCombine 2022-10-31 19:22:01 +08:00
Daniel Povey
b091ae5482 Add bias in weight module 2022-10-31 17:11:21 +08:00
Daniel Povey
5e51534fbc Introduce minimum probs in the SimpleCombiner 2022-10-31 17:02:21 +08:00
Daniel Povey
12f17f550e Introduce dropout rate to dynamic submodules of conformer. 2022-10-31 16:18:52 +08:00
Daniel Povey
3de8a5aef2 Bug fix 2022-10-31 15:50:46 +08:00
Daniel Povey
5fda800b6d Implement pooling module, add it after initial feedforward. 2022-10-31 15:49:18 +08:00
Daniel Povey
730e6c8914 Change schedule after initial loss not promising 2022-10-31 13:47:26 +08:00
Daniel Povey
b8db0f53f1 Change to schedule of bypass_scale min: make it larger, decrease slower. 2022-10-31 13:11:59 +08:00
Daniel Povey
efbb1d25c7 Restore the changes from scaled_adam_219 and scaled_adam_exp220, accidentally lost, re layer skipping 2022-10-30 14:59:49 +08:00
Daniel Povey
e4a22bbe96 Reduce initial clamp_min for bypass_scale from 1.0 to 0.5. 2022-10-30 14:43:02 +08:00
Daniel Povey
e9c69d8477 Add warmup schedule for zipformer encoder layer, from 1.0 -> 0.2. 2022-10-30 14:41:18 +08:00
Daniel Povey
8b0722e626 Rework how warmup count is produced; should not affect results. 2022-10-30 14:17:41 +08:00
Daniel Povey
6b6143f28c Merge branch 'scaled_adam_exp218' into scaled_adam_exp221 2022-10-30 13:17:29 +08:00
Daniel Povey
a3561c8dcd Have warmup schedule for layer-skipping 2022-10-29 21:01:00 +08:00
Daniel Povey
072776b2a1 Apply layer-skip dropout prob 2022-10-29 20:11:39 +08:00
Daniel Povey
9a7979d7b8 Avoid falling off the loop for weird inputs 2022-10-29 20:03:41 +08:00
Daniel Povey
05689f6354 Add skip connections as in normal U-net 2022-10-29 19:57:12 +08:00
Daniel Povey
bba454a349 Make decoder group size equal to 4. 2022-10-29 17:08:19 +08:00
Daniel Povey
ff03ec88a5 Tuning change to num encoder layers, inspired by relative param importance. 2022-10-29 15:56:02 +08:00
Daniel Povey
f995426324 Reduce min of bypass_scale from 0.5 to 0.3, and make it not applied in test mode. 2022-10-29 15:40:47 +08:00
Daniel Povey
435d0dec71 Reduce dim of linear positional encoding in attention layers. 2022-10-29 15:31:34 +08:00
Daniel Povey
96ea4cf1be Have 6 different encoder stacks, U-shaped network. 2022-10-28 20:36:45 +08:00
Daniel Povey
7b57a34227 Have 4 encoder stacks 2022-10-28 19:32:14 +08:00
Daniel Povey
de9a6ebd6c Fix bug RE seq lengths 2022-10-28 19:26:06 +08:00
Daniel Povey
d7d5188bd9 Refactor how the downsampling is done so that it happens later, but the 1st encoder stack still operates after a subsampling of 2. 2022-10-28 19:20:21 +08:00
Daniel Povey
0a89f51dc9 Have a 3rd encoder, at downsampling factor of 8. 2022-10-28 17:42:00 +08:00
Daniel Povey
ed1b4d5e5d Refactor zipformer for more flexibility so we can change number of encoder layers. 2022-10-28 17:32:38 +08:00
Daniel Povey
e592a920b4 Merge branch 'scaled_adam_exp198b' into scaled_adam_exp202 2022-10-28 13:13:55 +08:00
Daniel Povey
a067fe8026 Fix clamping of epsilon 2022-10-28 12:50:14 +08:00
Daniel Povey
7b8a0108ea Merge branch 'scaled_adam_exp188' into scaled_adam_exp198b 2022-10-28 12:49:36 +08:00
Daniel Povey
b9f6ba1aa2 Remove some unused variables. 2022-10-28 12:01:45 +08:00
Daniel Povey
c8abba75a9 Update decode.py by copying from pruned_transducer_stateless5 and changing directory name 2022-10-28 11:19:45 +08:00
Daniel Povey
5dfa141ca5 Rename Conformer to Zipformer 2022-10-27 22:43:46 +08:00
Daniel Povey
3f05e47447 Rename conformer.py to zipformer.py 2022-10-27 22:41:48 +08:00
Daniel Povey
be5c687fbd Merging upstream/master 2022-10-27 21:04:48 +08:00
Daniel Povey
f8c531cd23 Increase bypass_scale min from 0.4 to 0.5 2022-10-27 14:59:05 +08:00
Daniel Povey
2c400115e4 Increase bypass_scale from 0.2 to 0.4. 2022-10-27 14:30:46 +08:00
Daniel Povey
a7fc6ae38c Increase floor on bypass_scale from 0.1 to 0.2. 2022-10-27 14:09:34 +08:00
Daniel Povey
938510ac9f Fix clamping of bypass scale; remove a couple unused variables. 2022-10-27 14:05:53 +08:00
Daniel Povey
bf37c7ca85 Regularize how we apply the min and max to the eps of BasicNorm 2022-10-26 12:51:20 +08:00
Daniel Povey
a0507a83a5 Change scalar_max in optim.py from 2.0 to 5.0 2022-10-25 22:58:07 +08:00
Daniel Povey
78f3cba58c Add logging about memory used. 2022-10-25 19:19:33 +08:00
Daniel Povey
6a6df19bde Hopefully make penalize_abs_values_gt more memory efficient. 2022-10-25 18:41:33 +08:00
Daniel Povey
dbfbd8016b Cast to float16 in DoubleSwish forward 2022-10-25 13:16:00 +08:00
Daniel Povey
3159b09e8f Make 20 the limit for warmup_count 2022-10-25 12:58:27 +08:00
Daniel Povey
6ebff23cb9 Reduce cutoff from 100 to 5 for estimating OOM with warmup 2022-10-25 12:53:12 +08:00
Daniel Povey
9da5526659 Changes to more accurately estimate OOM conditions 2022-10-25 12:49:18 +08:00
Daniel Povey
1e8984174b Change to warmup schedule. 2022-10-25 12:27:00 +08:00
Daniel Povey
36cb279318 More memory efficient backprop for DoubleSwish. 2022-10-25 12:21:22 +08:00
Fangjun Kuang
499ac24ecb
Install kaldifst for GitHub actions (#632) 
* Install kaldifst for GitHub actions
 2022-10-24 15:07:29 +08:00