6 Commits

Author SHA1 Message Date
Daniel Povey
c3169222ae Simplified optimizer, rework somet things.. 2022-04-05 13:23:02 +08:00
Daniel Povey
47d49f29d7 Fix weight decay formula by adding 1/1-beta 2022-04-05 00:31:55 +08:00
Daniel Povey
2b0727a355 Fix weight decay formula by adding 1/1-beta 2022-04-05 00:31:28 +08:00
Daniel Povey
179d0605ea Change initialization to 0.25 2022-04-04 23:34:39 +08:00
Daniel Povey
d1f2f93460 Some fixes.. 2022-04-04 22:40:18 +08:00
Daniel Povey
72f4a673b1 First draft of new approach to learning rates + init 2022-04-04 20:21:34 +08:00