194 Commits

Author SHA1 Message Date
Daniel Povey
9bbf8ada57 Scale up diag of grad_cov by 1.0001 prior to diagonalizing it. 2022-08-06 07:06:23 +08:00
Daniel Povey
c021b4fec6 Increase cov_min[3] from 0.0001 to 0.001 2022-08-06 07:02:52 +08:00
Daniel Povey
a5b9b7b974 Cosmetic changes 2022-08-05 03:51:00 +08:00
Daniel Povey
dc9133227f Reworked how inverse is done, fixed bug in _apply_min_max_with_metric, regarding how M is normalized. 2022-08-04 09:46:14 +08:00
Daniel Povey
766bf69a98 Reduce cov_max[2] from 4.0 to 3.5 2022-08-03 04:10:11 +08:00
Daniel Povey
129b28aa9b Increase cov_min[2] from 0.05 to 0.1; decrease cov_max[2] from 5.0 to 4.0. 2022-08-02 15:17:24 +08:00
Daniel Povey
202752418a Increase cov_min[2] from 0.02 to 0.05. 2022-08-02 15:15:41 +08:00
Daniel Povey
e44ab25e99 Bug fix 2022-08-02 14:31:37 +08:00
Daniel Povey
e9f4ada1c0 Swap the order of applying min and max in smoothing operations 2022-08-02 11:55:43 +08:00
Daniel Povey
9473c7e23d Lots of changes to how min and max are applied, use 1-norm for min in smooth_cov but not _apply_min_max_with_metric. 2022-08-02 11:29:54 +08:00
Daniel Povey
6ab4cf615d 1st draft of new method of normalizing covs that uses normalization w.r.t. spectral 2-norm 2022-08-02 09:34:37 +08:00
Daniel Povey
4919134a94 Merge making hidden_dim an arg 2022-08-02 09:09:29 +08:00
Daniel Povey
c64bd5ebcd Merge making hidden_dim an arg 2022-08-02 09:07:36 +08:00
Daniel Povey
b008340d83 Merge making hidden_dim an arg 2022-08-02 09:01:19 +08:00
Daniel Povey
9f2229edb5 Merge making hidden_dim an arg 2022-08-02 08:58:00 +08:00
Daniel Povey
a45f820e25 Merge making hidden_dim an arg 2022-08-02 08:56:36 +08:00
Daniel Povey
804f264ffd Merge hidden_dim providing it as arg 2022-08-02 08:40:13 +08:00
Daniel Povey
ee311247ea Decrease debugging freq 2022-08-01 03:55:18 +08:00
Daniel Povey
4c5d49c448 Some numerical improvements, and a fix to calculation of mean_eig in _apply_min_max_with_metric(), to average over blocks too. 2022-08-01 03:51:39 +08:00
Daniel Povey
e2cc09a8c6 Fix issue with max_eig formula; restore cov_min[1]=0.0025. 2022-07-31 18:29:44 +08:00
Daniel Povey
3590c2fc42 Set cov_min[1] to 0 to stop an invertibility problem 2022-07-31 18:06:01 +08:00
Daniel Povey
7231c610e8 Restore min_cov applied with G. 2022-07-31 02:22:07 -07:00
Daniel Povey
d84a2e22e3 Applying max to G with noinv method with metric. 2022-07-31 02:10:27 -07:00
Daniel Povey
2042c9862c Merge branch 'pradam_exp1m4_nophase1_noinv' into pradam_exp1m4_nophase1_rework_noinv 2022-07-31 01:32:36 -07:00
Daniel Povey
90fa8a63eb Use different approach for applying max eig, with matmul, no inverse. 2022-07-31 01:32:11 -07:00
Daniel Povey
ed1a147ef1 Implement no-inverse max-cov 2022-07-31 00:08:02 -07:00
Daniel Povey
0666789cb8 Small numerical improvements; config change of eps and G_diag changed 1.01 to 1.005; decrease an eps from 1e-10 to 1e-20 2022-07-30 21:48:54 -07:00
Daniel Povey
cb67540cdc this version not working great 2022-07-30 21:14:03 -07:00
Daniel Povey
790e8c4ba9 Changes that should not really affect the results, just cleanup. 2022-07-30 19:20:36 -07:00
Daniel Povey
5184ac570d Removing phase1, adding regular smoothing with the mean. 2022-07-30 19:15:51 -07:00
Daniel Povey
105d49d31b Big simplification to update rule 2022-07-30 00:21:12 -07:00
Daniel Povey
4d5323d6a3 Change cov_min to intermediate value, not seeing clear diffs from changing it 2022-07-30 09:06:23 +08:00
Daniel Povey
c0e652ea87 Fix code, was smoothing in wrong basis. 2022-07-30 09:01:11 +08:00
Daniel Povey
17bc002e6e Refactoring that does not affect results. 2022-07-30 07:45:29 +08:00
Daniel Povey
3110138ab5 Smooth grad_cov with eps; add a 4th stage of smoothing, this time on Z_inv. 2022-07-30 07:30:32 +08:00
Daniel Povey
3ad042444e More changes to reduce numerical roundoff for dims with zero grad and params. 2022-07-29 14:38:50 +08:00
Daniel Povey
3c1fddaf48 Rework computation to reduce numerical roundoff 2022-07-29 06:22:17 +08:00
Daniel Povey
0d038a6ea4 Remove debugging statement 2022-07-28 09:26:11 +08:00
Daniel Povey
8654a7385d Add denom_rel_eps, and set it to 1e-05 2022-07-28 09:10:20 +08:00
Daniel Povey
dc565f729b Take into account various outcomes from parameter tuning 2022-07-28 09:06:59 +08:00
Daniel Povey
fe595f8772 Improve debugging output. 2022-07-25 09:02:36 +08:00
Daniel Povey
854c2965a9 Fix bug regarding G_prime being zero 2022-07-25 06:57:52 +08:00
Daniel Povey
3acdf3b395 Reworking the computation of Z to be numerically better. 2022-07-25 06:37:26 +08:00
Daniel Povey
5513f7fee5 Initial version of fixing numerical issue, will continue though 2022-07-25 06:27:01 +08:00
Daniel Povey
06718052ec Refactoring, putting tunable values in constructor, a little cleanup 2022-07-25 04:31:42 +08:00
Daniel Povey
8efc512823 Remove some debugging code, found the mismatch 2022-07-24 11:52:10 +08:00
Daniel Povey
ba96439c76 Saving version I am trying to debug 2022-07-24 11:00:40 +08:00
Daniel Povey
962e95f119 Using a more flexible test. Moved to simpler update , tuned diffrently. 2022-07-24 09:20:53 +08:00
Daniel Povey
b8a9485011 Print git version for test output 2022-07-24 06:54:29 +08:00
Daniel Povey
48ac7e0bc3 Add max as well as min to G_prime 2022-07-24 06:50:05 +08:00