79 Commits

Author	SHA1	Message	Date
Daniel Povey	923468b8af	Deal with SVD failure better.	2022-07-08 09:00:12 +08:00
Daniel Povey	97feb8a3ec	Reduce meta_lr_scale, reduces loss @140 from 1.4 to 0.39	2022-07-08 06:33:07 +08:00
Daniel Povey	b6199a71e9	Introduce delta_scale to slow down changes on M; significantly better.	2022-07-08 06:05:31 +08:00
Daniel Povey	ceb9815f2b	Increase lr_est_period	2022-07-08 05:51:18 +08:00
Daniel Povey	fb36712e6b	Another bug fix, regarding Q being transposed.	2022-07-08 05:22:24 +08:00
Daniel Povey	ad2e698fc3	Cleanups	2022-07-08 04:44:21 +08:00
Daniel Povey	04d2e10b4f	Version that runs	2022-07-08 04:37:46 +08:00
Daniel Povey	e6d00ee3e4	More drafts of new method, not tested.	2022-07-06 23:05:06 -07:00
Daniel Povey	26815d177f	Draft of the new method..	2022-07-06 22:59:36 -07:00
Daniel Povey	e9e2a85c95	In the middle of reworking for new idea	2022-07-06 13:35:19 -07:00
Daniel Povey	41368f6b63	Change comment	2022-07-05 17:11:45 -07:00
Daniel Povey	2692d5f903	Closer to finished	2022-06-30 23:54:26 -07:00
Daniel Povey	d64cb1cb48	draft, not working, will edit locally	2022-06-30 15:35:26 +08:00
Daniel Povey	0b811546f3	Apply reverse_cutoff with param_pow, to make it not too strong.	2022-06-26 11:07:08 +08:00
Daniel Povey	0aa5a334d6	Fix regarding reverse_cutoff formula	2022-06-25 18:24:05 +08:00
Daniel Povey	8a0277d493	Increase param_reverse_cutoff from 4 to 16	2022-06-25 18:02:00 +08:00
Daniel Povey	2ccc7ccbeb	Make the application of param_reverse_cutoff conditional, not applied when rank is a problem	2022-06-25 17:56:04 +08:00
Daniel Povey	146d7c5a93	Bug fix	2022-06-24 19:49:13 +08:00
Daniel Povey	117d348f70	This time get it right, last time was starting from wrong base.	2022-06-24 19:00:58 +08:00
Daniel Povey	6002e19836	Change max_fullcov_size back to 1023.. doing it this way because I forgot to commit the changes on this branch	2022-06-24 18:59:34 +08:00
Daniel Povey	c2ec901e73	Change param_pow from 0.75 to 0.5	2022-06-24 17:14:41 +08:00
Daniel Povey	f327407308	Avoid error if svd fails	2022-06-24 13:20:20 +08:00
Daniel Povey	2233c852fd	Increase max_fullcov_size from 1023 to 2048	2022-06-24 13:02:17 +08:00
Daniel Povey	683b8e1504	Some code reworking and fixes, rationalizing how speedup is done and fix an issue affecting learning rate.	2022-06-23 15:40:51 +08:00
Daniel Povey	c34344e98f	Change param_pow,grad_pow to 0.75,0.95	2022-06-23 14:04:11 +08:00
Daniel Povey	3767f3d616	Reducd param_pow from 0.8 to 0.7	2022-06-23 11:12:12 +08:00
Daniel Povey	ca17f23da3	Reduce powers to 0.8,0.85	2022-06-23 11:09:27 +08:00
Daniel Povey	09282ca28c	Reduce some print statements, convert some to info	2022-06-23 10:41:08 +08:00
Daniel Povey	7f756b2910	Introduce grad_pow option, set it to 0.9 and param_pow to 0.85	2022-06-23 10:33:14 +08:00
Daniel Povey	4124cd7241	Reduce debug frequencies	2022-06-20 13:48:42 +08:00
Daniel Povey	c10a9889fa	Reduce grad_eps to 1e-14	2022-06-20 13:45:53 +08:00
Daniel Povey	b560221eea	Change param_pow to 0.75	2022-06-20 00:29:07 +08:00
Daniel Povey	cf816a9138	Change param_pow to 0.0	2022-06-19 22:44:17 +08:00
Daniel Povey	ad5337fe95	Set param_pow to 0.5	2022-06-19 22:14:34 +08:00
Daniel Povey	8acc398428	Reduce debug freq	2022-06-19 11:21:48 +08:00
Daniel Povey	29dfe37f22	Implement min,max param rms	2022-06-18 22:22:23 +08:00
Daniel Povey	3fb3cc4e23	Make random cov have correct diagonal; reset grad_min_rand to 0.	2022-06-18 17:04:44 +08:00
Daniel Povey	3f0b57403e	Min-rand-scale 0.2 for grad cov	2022-06-18 16:26:37 +08:00
Daniel Povey	e996f7f371	First version I am running, of the speedup code	2022-06-18 15:17:51 +08:00
Daniel Povey	a6bf32c3e0	Fast code seems to pass tests.	2022-06-18 12:56:16 +08:00
Daniel Povey	cbccc1dd91	Fix bias_correction2, power of 0.5	2022-06-17 20:20:16 +08:00
Daniel Povey	2d13b682fe	Revert param_rel_eps to old value	2022-06-17 18:22:27 +08:00
Daniel Povey	92fb1c52c8	Reduce debug statement frequency	2022-06-17 16:03:50 +08:00
Daniel Povey	d9a3eba370	Fix bug that caused it to use too much memory	2022-06-17 16:02:47 +08:00
Daniel Povey	6873aa1a13	Bug fix to print statement	2022-06-17 15:35:07 +08:00
Daniel Povey	90622f88ad	Decrease param_rel_eps default from 1e-4 to 1e-2	2022-06-17 15:10:46 +08:00
Daniel Povey	2fe4af8c99	Prevent crash due to error in C_diff	2022-06-17 12:35:55 +08:00
Daniel Povey	6c499dcd66	Augment debug output	2022-06-17 12:32:09 +08:00
Daniel Povey	5c3e4f506f	Reduce debug frequency	2022-06-17 12:17:48 +08:00
Daniel Povey	47df144253	Bug fix for scalar case	2022-06-17 12:16:12 +08:00