188 Commits

Author	SHA1	Message	Date
Daniel Povey	d1e4ae788d	Refactor how learning rate is set.	2022-04-10 15:25:27 +08:00
Daniel Povey	82d58629ea	Implement 2p version of learning rate schedule.	2022-04-10 13:50:31 +08:00
Daniel Povey	da50525ca5	Make lrate rule more symmetric	2022-04-10 13:25:40 +08:00
Daniel Povey	4d41ee0caa	Implement 2o schedule	2022-04-09 18:37:03 +08:00
Daniel Povey	db72aee1f0	Set 2n rule..	2022-04-09 18:15:56 +08:00
Daniel Povey	0f8ee68af2	Fix bug	2022-04-08 16:53:42 +08:00
Daniel Povey	f587cd527d	Change exponential part of lrate to be epoch based	2022-04-08 16:24:21 +08:00
Daniel Povey	6ee32cf7af	Set new scheduler	2022-04-08 16:10:06 +08:00
Daniel Povey	61486a0f76	Remove initial_speed	2022-04-06 13:17:26 +08:00
Daniel Povey	a41e93437c	Change some defaults in LR-setting rule.	2022-04-06 12:36:58 +08:00
Daniel Povey	2545237eb3	Changing initial_speed from 0.25 to 01	2022-04-05 18:00:54 +08:00
Daniel Povey	25724b5ce9	Bug-fix RE sign of target_rms	2022-04-05 13:49:35 +08:00
Daniel Povey	d1a669162c	Fix bug in lambda	2022-04-05 13:31:52 +08:00
Daniel Povey	ed8eba91e1	Reduce model_warm_step from 4k to 3k	2022-04-05 13:24:09 +08:00
Daniel Povey	c3169222ae	Simplified optimizer, rework somet things..	2022-04-05 13:23:02 +08:00
Daniel Povey	0f5957394b	Fix to reading scheudler from optim	2022-04-05 12:58:43 +08:00
Daniel Povey	1548cc7462	Fix checkpoint-writing	2022-04-05 11:19:40 +08:00
Daniel Povey	47d49f29d7	Fix weight decay formula by adding 1/1-beta	2022-04-05 00:31:55 +08:00
Daniel Povey	2b0727a355	Fix weight decay formula by adding 1/1-beta	2022-04-05 00:31:28 +08:00
Daniel Povey	234366e51c	Fix type of parameter	2022-04-05 00:18:36 +08:00
Daniel Povey	179d0605ea	Change initialization to 0.25	2022-04-04 23:34:39 +08:00
Daniel Povey	d1f2f93460	Some fixes..	2022-04-04 22:40:18 +08:00
Daniel Povey	72f4a673b1	First draft of new approach to learning rates + init	2022-04-04 20:21:34 +08:00
Daniel Povey	4929e4cf32	Change how warm-step is set	2022-04-04 17:09:25 +08:00
Daniel Povey	a5bbcd7b71	Make training more efficient, avoid redoing some projections.	2022-04-04 14:14:03 +08:00
Daniel Povey	0fd0828f79	Fix to joiner to allow different dims	2022-04-04 13:34:43 +08:00
Daniel Povey	807fcada68	Change learning speed of simple_lm_proj	2022-04-02 20:15:11 +08:00
Daniel Povey	34500afc43	Various bug fixes	2022-04-02 20:06:43 +08:00
Daniel Povey	8be10d3d6c	First draft of model rework	2022-04-02 20:03:21 +08:00
Daniel Povey	eec597fdd5	Merge changes from master	2022-04-02 18:45:20 +08:00
Daniel Povey	e0ba4ef3ec	Make layer dropout rate 0.075, was 0.1.	2022-04-02 17:48:54 +08:00
Daniel Povey	45f872c27d	Remove final dropout	2022-04-01 19:33:20 +08:00
Daniel Povey	92ec2e356e	Fix test-mode	2022-04-01 12:22:12 +08:00
Daniel Povey	8caa18e2fe	Bug fix to warmup_scale	2022-03-31 17:30:51 +08:00
Daniel Povey	49bc761ba1	Merge branch 'rework2i_restoredrop_scaled_warmup' into rework2i_restoredrop_scaled_warmup_2proj ... # Conflicts: # egs/librispeech/ASR/pruned_transducer_stateless2/model.py	2022-03-31 14:45:55 +08:00
Daniel Povey	e663713258	Change how warmup is applied.	2022-03-31 14:43:49 +08:00
Daniel Povey	fcb0dba2cf	Reduce initial_speed from 0.5 to 0.25	2022-03-31 13:47:28 +08:00
Daniel Povey	025d690995	Reduce initial_speed further from 0.5 to 0.25	2022-03-31 13:39:56 +08:00
Daniel Povey	ec54fa85cc	Use initial_speed=0.5	2022-03-31 13:04:09 +08:00
Daniel Povey	e59db01b7c	Reduce initial_speed	2022-03-31 13:03:26 +08:00
Daniel Povey	c67ae0f3a1	Make 2 projections..	2022-03-31 13:02:40 +08:00
Daniel Povey	f75d40c725	Replace nn.Linear with ScaledLinear in simple joiner	2022-03-31 12:18:31 +08:00
Daniel Povey	9a0c2e7fee	Merge branch 'rework2i' into rework2i_restoredrop	2022-03-31 12:17:02 +08:00
Daniel Povey	f47fe8337a	Remove some un-used code	2022-03-31 12:16:08 +08:00
Daniel Povey	0599f38281	Add final dropout to conformer	2022-03-31 11:53:54 +08:00
Daniel Povey	a2aca9f643	Bug-fix	2022-03-30 21:42:15 +08:00
Daniel Povey	f87811e65c	Fix RE identity	2022-03-30 21:41:46 +08:00
Daniel Povey	709c387ce6	Initial refactoring to remove unnecessary vocab_size	2022-03-30 21:40:22 +08:00
Daniel Povey	74121ac478	Merge branch 'rework2h_randloader_pow0.333_conv_8' into rework2h_randloader_pow0.333_conv_8_lessdrop_speed ... # Conflicts: # egs/librispeech/ASR/pruned_transducer_stateless2/conformer.py	2022-03-30 12:24:15 +08:00
Daniel Povey	37ab0bcfa5	Reduce speed of some components	2022-03-30 11:46:23 +08:00