183 Commits

Author	SHA1	Message	Date
Daniel Povey	4919134a94	Merge making hidden_dim an arg	2022-08-02 09:09:29 +08:00
Daniel Povey	c64bd5ebcd	Merge making hidden_dim an arg	2022-08-02 09:07:36 +08:00
Daniel Povey	b008340d83	Merge making hidden_dim an arg	2022-08-02 09:01:19 +08:00
Daniel Povey	9f2229edb5	Merge making hidden_dim an arg	2022-08-02 08:58:00 +08:00
Daniel Povey	a45f820e25	Merge making hidden_dim an arg	2022-08-02 08:56:36 +08:00
Daniel Povey	804f264ffd	Merge hidden_dim providing it as arg	2022-08-02 08:40:13 +08:00
Daniel Povey	ee311247ea	Decrease debugging freq	2022-08-01 03:55:18 +08:00
Daniel Povey	4c5d49c448	Some numerical improvements, and a fix to calculation of mean_eig in _apply_min_max_with_metric(), to average over blocks too.	2022-08-01 03:51:39 +08:00
Daniel Povey	e2cc09a8c6	Fix issue with max_eig formula; restore cov_min[1]=0.0025.	2022-07-31 18:29:44 +08:00
Daniel Povey	3590c2fc42	Set cov_min[1] to 0 to stop an invertibility problem	2022-07-31 18:06:01 +08:00
Daniel Povey	7231c610e8	Restore min_cov applied with G.	2022-07-31 02:22:07 -07:00
Daniel Povey	d84a2e22e3	Applying max to G with noinv method with metric.	2022-07-31 02:10:27 -07:00
Daniel Povey	2042c9862c	Merge branch 'pradam_exp1m4_nophase1_noinv' into pradam_exp1m4_nophase1_rework_noinv	2022-07-31 01:32:36 -07:00
Daniel Povey	90fa8a63eb	Use different approach for applying max eig, with matmul, no inverse.	2022-07-31 01:32:11 -07:00
Daniel Povey	ed1a147ef1	Implement no-inverse max-cov	2022-07-31 00:08:02 -07:00
Daniel Povey	0666789cb8	Small numerical improvements; config change of eps and G_diag changed 1.01 to 1.005; decrease an eps from 1e-10 to 1e-20	2022-07-30 21:48:54 -07:00
Daniel Povey	cb67540cdc	this version not working great	2022-07-30 21:14:03 -07:00
Daniel Povey	790e8c4ba9	Changes that should not really affect the results, just cleanup.	2022-07-30 19:20:36 -07:00
Daniel Povey	5184ac570d	Removing phase1, adding regular smoothing with the mean.	2022-07-30 19:15:51 -07:00
Daniel Povey	105d49d31b	Big simplification to update rule	2022-07-30 00:21:12 -07:00
Daniel Povey	4d5323d6a3	Change cov_min to intermediate value, not seeing clear diffs from changing it	2022-07-30 09:06:23 +08:00
Daniel Povey	c0e652ea87	Fix code, was smoothing in wrong basis.	2022-07-30 09:01:11 +08:00
Daniel Povey	17bc002e6e	Refactoring that does not affect results.	2022-07-30 07:45:29 +08:00
Daniel Povey	3110138ab5	Smooth grad_cov with eps; add a 4th stage of smoothing, this time on Z_inv.	2022-07-30 07:30:32 +08:00
Daniel Povey	3ad042444e	More changes to reduce numerical roundoff for dims with zero grad and params.	2022-07-29 14:38:50 +08:00
Daniel Povey	3c1fddaf48	Rework computation to reduce numerical roundoff	2022-07-29 06:22:17 +08:00
Daniel Povey	0d038a6ea4	Remove debugging statement	2022-07-28 09:26:11 +08:00
Daniel Povey	8654a7385d	Add denom_rel_eps, and set it to 1e-05	2022-07-28 09:10:20 +08:00
Daniel Povey	dc565f729b	Take into account various outcomes from parameter tuning	2022-07-28 09:06:59 +08:00
Daniel Povey	fe595f8772	Improve debugging output.	2022-07-25 09:02:36 +08:00
Daniel Povey	854c2965a9	Fix bug regarding G_prime being zero	2022-07-25 06:57:52 +08:00
Daniel Povey	3acdf3b395	Reworking the computation of Z to be numerically better.	2022-07-25 06:37:26 +08:00
Daniel Povey	5513f7fee5	Initial version of fixing numerical issue, will continue though	2022-07-25 06:27:01 +08:00
Daniel Povey	06718052ec	Refactoring, putting tunable values in constructor, a little cleanup	2022-07-25 04:31:42 +08:00
Daniel Povey	8efc512823	Remove some debugging code, found the mismatch	2022-07-24 11:52:10 +08:00
Daniel Povey	ba96439c76	Saving version I am trying to debug	2022-07-24 11:00:40 +08:00
Daniel Povey	962e95f119	Using a more flexible test. Moved to simpler update , tuned diffrently.	2022-07-24 09:20:53 +08:00
Daniel Povey	b8a9485011	Print git version for test output	2022-07-24 06:54:29 +08:00
Daniel Povey	48ac7e0bc3	Add max as well as min to G_prime	2022-07-24 06:50:05 +08:00
Daniel Povey	6290fcb535	Cleanup and refactoring	2022-07-24 05:48:38 +08:00
Daniel Povey	8a9bbb93bc	Cosmetic fixes	2022-07-24 04:45:57 +08:00
Daniel Povey	966ac36cde	Fixes to comments	2022-07-24 04:36:41 +08:00
Daniel Povey	33ffd17515	Some cleanup	2022-07-24 04:22:11 +08:00
Daniel Povey	ddceb7963b	Interpolate between iterative estimate of scale, and original value.	2022-07-23 15:27:48 +08:00
Daniel Povey	2c4bdd0ad0	Add _update_param_scales_simple(), add documentation	2022-07-23 14:49:58 +08:00
Daniel Povey	9730352257	Redce smoothing constant slightly	2022-07-23 13:12:31 +08:00
Daniel Povey	e1873fc0bb	Tune phase2 again, from 0.005,5.0 to 0.01,40. Epoch 140 is 0.21/0.149	2022-07-23 10:10:01 +08:00
Daniel Povey	0fc58bac56	More tuning, epoch-140 results are 0.23,0.11	2022-07-23 09:52:51 +08:00
Daniel Povey	34a2d331bf	Smooth in opposite orientation to G	2022-07-23 09:38:16 +08:00
Daniel Povey	a972655a70	Tuning.	2022-07-23 09:15:49 +08:00