8 Commits

Author	SHA1	Message	Date
Daniel Povey	a41e93437c	Change some defaults in LR-setting rule.	2022-04-06 12:36:58 +08:00
Daniel Povey	25724b5ce9	Bug-fix RE sign of target_rms	2022-04-05 13:49:35 +08:00
Daniel Povey	c3169222ae	Simplified optimizer, rework somet things..	2022-04-05 13:23:02 +08:00
Daniel Povey	47d49f29d7	Fix weight decay formula by adding 1/1-beta	2022-04-05 00:31:55 +08:00
Daniel Povey	2b0727a355	Fix weight decay formula by adding 1/1-beta	2022-04-05 00:31:28 +08:00
Daniel Povey	179d0605ea	Change initialization to 0.25	2022-04-04 23:34:39 +08:00
Daniel Povey	d1f2f93460	Some fixes..	2022-04-04 22:40:18 +08:00
Daniel Povey	72f4a673b1	First draft of new approach to learning rates + init	2022-04-04 20:21:34 +08:00