646 Commits

Author	SHA1	Message	Date
Daniel Povey	2fe4af8c99	Prevent crash due to error in C_diff	2022-06-17 12:35:55 +08:00
Daniel Povey	6c499dcd66	Augment debug output	2022-06-17 12:32:09 +08:00
Daniel Povey	5c3e4f506f	Reduce debug frequency	2022-06-17 12:17:48 +08:00
Daniel Povey	47df144253	Bug fix for scalar case	2022-06-17 12:16:12 +08:00
Daniel Povey	9e92d13a33	Some bug fixes.. seems to be working.	2022-06-17 12:04:21 +08:00
Daniel Povey	827a37c7fc	Closer to finished	2022-06-17 11:10:43 +08:00
Daniel Povey	94e48ff07e	Some code improvements...	2022-06-16 10:43:26 +08:00
Daniel Povey	f9bb745cf3	Use exp_avg_sq as ref_exp_avg_sq	2022-06-15 14:03:26 +08:00
Daniel Povey	860322bf30	Make it less verbose; fix scale_speed setting; testing min_diag_smooth=1.0 for debug	2022-06-15 13:02:04 +08:00
Daniel Povey	bc5d72b0f3	Fix potential bug re scale change	2022-06-15 12:49:47 +08:00
Daniel Povey	57957cc049	Add diagnostics	2022-06-15 12:39:16 +08:00
Daniel Povey	0679b363b0	Some debug stuff, modify cond_eps for param_cov	2022-06-14 22:38:51 +08:00
Daniel Povey	932cedce59	Bug fix RE eps*eps; add/tune diagnostics	2022-06-14 21:43:27 +08:00
Daniel Povey	2615e48779	Some cleanup and configuration changes.	2022-06-14 21:17:50 +08:00
Daniel Povey	c6f62dfc08	Version that is passing the tests	2022-06-14 20:45:57 +08:00
Daniel Povey	e597b68160	Improve roundoff properties of algorithm for getting C, use randomized testing.	2022-06-14 12:59:41 +08:00
Daniel Povey	d729e970e1	Some changes for stability	2022-06-14 10:25:17 +08:00
Daniel Povey	d1e96afce2	Improve speed by messing with configuration, removing asserts.	2022-06-13 17:27:34 +08:00
Daniel Povey	a41c4b6c9b	Bug fixes	2022-06-13 17:20:46 +08:00
Daniel Povey	41b0e582df	Bug fix to typo in optim.py	2022-06-13 16:10:49 +08:00
Daniel Povey	ca7cffcb42	Remove Decorrelate() class	2022-06-13 16:08:32 +08:00
Daniel Povey	7338c60296	Remove Decorrelate()	2022-06-13 16:07:15 +08:00
Daniel Povey	c1f487e36d	Move optim2.py to optim.py; use this optimizer in train.py	2022-06-13 16:05:46 +08:00
Daniel Povey	4e0d8c45f4	Move optim2.py to optim.py; use this optimizer in train.py	2022-06-13 16:05:22 +08:00
Daniel Povey	d5b90f278e	Adding draft of new optimizer	2022-06-13 15:01:10 +08:00
Daniel Povey	8c4b5454e7	Remove unnecessary line	2022-06-11 13:14:18 +08:00
Daniel Povey	73fd2fcd77	Updates to cain to new param_rms per element	2022-06-11 13:08:08 +08:00
Daniel Povey	d301f8ac6c	Merge Decorrelate work, and simplification to RandomCombine, into pruned_transducer_stateless7	2022-06-11 11:07:07 +08:00
Daniel Povey	30379bb35d	Merge branch 'cain3d_merge_supersimple' into merge_decorr_cain	2022-06-11 10:51:30 +08:00
Daniel Povey	a3c97554e8	Reduce scale from 0.1 to 0.05	2022-06-10 23:34:46 +08:00
Daniel Povey	2bbc63a2f5	Change first1k to decay1k	2022-06-10 23:33:16 +08:00
Daniel Povey	41cdbae753	Increase scale from 0.05 to 0.1	2022-06-10 19:07:48 +08:00
Daniel Povey	5fb64a59b8	Change beta from 0.8 to 0.95	2022-06-10 19:05:04 +08:00
Daniel Povey	6ed181595b	Scale by grad norm	2022-06-10 18:34:42 +08:00
Daniel Povey	6a47bf1178	Decrease scale to 0.05	2022-06-10 18:33:50 +08:00
Daniel Povey	9d4633facf	Simplified gradient scaling [no scaling]; only use 1k first iters; beta =0.8	2022-06-10 17:16:05 +08:00
Daniel Povey	cecd52155c	Apply x row scaling with grad	2022-06-10 16:42:52 +08:00
Daniel Povey	86c2d0fcc0	Make it run only for first 3k steps; larger scale; remove limit of 1.0	2022-06-10 16:33:22 +08:00
Daniel Povey	eeb95ed502	Fix issue with cov scale	2022-06-10 16:25:45 +08:00
Daniel Povey	c671e213fc	Increase beta from 0.95 to 0.98	2022-06-10 14:39:58 +08:00
Daniel Povey	c6cfb1e5fa	Remove logging that was excessive	2022-06-10 14:25:23 +08:00
Daniel Povey	ff0309947a	Do scaling a different way, so loss function is more consistent; accum stats in backward pass	2022-06-10 14:16:44 +08:00
Daniel Povey	58cbc3d961	Move PseudoNormalizeFunction to a different place.	2022-06-10 14:01:13 +08:00
Daniel Povey	77357cdaa8	Change scale from 0.1 to 0.05	2022-06-10 12:42:43 +08:00
Daniel Povey	173c8e6ca3	Make decorrelate scale really 0.1	2022-06-10 12:39:01 +08:00
Daniel Povey	42667aacf9	Remove linear layers from RandomCombine	2022-06-10 11:38:50 +08:00
Daniel Povey	c92d9d72aa	Fix inf issue	2022-06-10 11:20:47 +08:00
Daniel Povey	950cd4a3e8	Introduce normalization..	2022-06-10 10:47:18 +08:00
Daniel Povey	e891a65735	Move Decorrelate module to after encoder, with scale 0.02->0.1	2022-06-10 10:15:28 +08:00
Daniel Povey	4a5143e548	Increase decay to 1k	2022-06-10 10:09:46 +08:00