196 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Daniel Povey	137eae0b95	Reduce max_factor to 0.01	2022-03-11 14:42:17 +08:00
Daniel Povey	ab9a17413a	Scale up pos_bias_u and pos_bias_v before use.	2022-03-11 14:37:52 +08:00
Daniel Povey	e3e14cf7a4	Change min-abs threshold from 0.2 to 0.5	2022-03-11 14:16:33 +08:00
Daniel Povey	bfce5f63e4	Fix dirname	2022-03-10 23:49:09 +08:00
Daniel Povey	76560f255c	Add min-abs-value 0.2	2022-03-10 23:48:46 +08:00
Daniel Povey	2fa9c636a4	use nonzero threshold in DerivBalancer	2022-03-10 23:24:55 +08:00
Daniel Povey	425e274c82	Replace norm in ConvolutionModule with a scaling factor.	2022-03-10 16:01:53 +08:00
Daniel Povey	87b843f023	Change exp dir	2022-03-10 14:44:55 +08:00
Daniel Povey	b55472bb42	Replace most normalizations with scales (still have norm in conv)	2022-03-10 14:43:54 +08:00
Daniel Povey	059b57ad37	Add BasicNorm module	2022-03-10 14:32:05 +08:00
Daniel Povey	feb20ca84d	Merge changes to diagnostics	2022-03-10 10:31:42 +08:00
Daniel Povey	1e5455ba29	Update diagnostics	2022-03-10 10:28:48 +08:00
Daniel Povey	d074cf73c6	Extensions to diagnostics code	2022-03-09 20:37:20 +08:00
Daniel Povey	e2ace9d545	Replace norm on input layer with scale of 0.1.	2022-03-07 11:24:04 +08:00
Daniel Povey	a37d98463a	Restore ConvolutionModule to state before changes; change all Swish,Swish(Swish) to SwishOffset.	2022-03-06 11:55:02 +08:00
Daniel Povey	8a8b81cd18	Replace relu with swish-squared.	2022-03-05 22:21:42 +08:00
Daniel Povey	5f2c0a09b7	Convert swish nonlinearities to ReLU	2022-03-05 16:28:24 +08:00
Daniel Povey	0cd14ae739	Fix exp dir	2022-03-05 12:17:09 +08:00
Daniel Povey	65b09dd5f2	Double the threshold in brelu; slightly increase max_factor.	2022-03-05 00:07:14 +08:00
Daniel Povey	74f2b163de	Merge diagnostics improvement	2022-03-04 23:15:47 +08:00
Daniel Povey	6252282fd0	Add deriv-balancing code	2022-03-04 20:19:11 +08:00
Daniel Povey	eb3ed54202	Reduce scale from 50 to 20	2022-03-04 15:56:45 +08:00
Daniel Povey	9cc5999829	Fix duplicate Swish; replace norm+swish with swish+exp-scale in convolution module	2022-03-04 15:50:51 +08:00
Daniel Povey	7e88999641	Increase scale from 20 to 50.	2022-03-04 14:31:29 +08:00
Daniel Povey	3207bd98a9	Increase scale on Scale from 4 to 20	2022-03-04 13:16:40 +08:00
Daniel Povey	503f8d521c	Fix bug in diagnostics	2022-03-04 13:08:56 +08:00
Daniel Povey	3d9ddc2016	Fix backprop bug	2022-03-04 12:29:44 +08:00
Daniel Povey	cd216f50b6	Add import	2022-03-04 11:03:01 +08:00
Daniel Povey	bc6c720e25	Combine ExpScale and swish for memory reduction	2022-03-04 10:52:05 +08:00
Daniel Povey	23b3aa233c	Double learning rate of exp-scale units	2022-03-04 00:42:37 +08:00
Daniel Povey	5c177fc52b	pelu_base->expscale, add 2xExpScale in subsampling, and in feedforward units.	2022-03-03 23:52:03 +08:00
Daniel Povey	3fb559d2f0	Add baseline for the PeLU expt, keeping only the small normalization-related changes.	2022-03-02 18:27:08 +08:00
Daniel Povey	9ed7d55a84	Small bug fixes/imports	2022-03-02 16:34:55 +08:00
Daniel Povey	9d1b4ae046	Add pelu to this good-performing setup..	2022-03-02 16:33:27 +08:00
Daniel Povey	2ff520c800	Improvements to diagnostics (RE those with 1 dim	2022-02-28 12:22:27 +08:00
Daniel Povey	c1063def95	First version of rand-combine iterated-training-like idea.	2022-02-27 17:34:58 +08:00
Daniel Povey	63d8d935d4	Refactor/simplify ConformerEncoder	2022-02-27 13:56:15 +08:00
Daniel Povey	581786a6d3	Adding diagnostics code...	2022-02-27 13:44:43 +08:00
Daniel Povey	2af1b3af98	Remove ReLU in attention	2022-02-14 19:39:19 +08:00
Daniel Povey	d187ad8b73	Change max_frames from 0.2 to 0.15	2022-02-11 16:24:17 +08:00
Daniel Povey	4cd2c02fff	Fix num_time_masks code; revert 0.8 to 0.9	2022-02-10 15:53:11 +08:00
Daniel Povey	c170c53006	Change p=0.9 to p=0.8 in SpecAug	2022-02-10 14:59:14 +08:00
Daniel Povey	8aa50df4f0	Change p=0.5->0.9, mask_fraction 0.3->0.2	2022-02-09 22:52:53 +08:00
Daniel Povey	dd19a6a2b1	Fix to num_feature_masks bug I introduced; reduce max_frames_mask_fraction 0.4->0.3	2022-02-09 12:02:19 +08:00
Daniel Povey	bd36216e8c	Use much more aggressive SpecAug setup	2022-02-08 21:55:20 +08:00
Daniel Povey	beaf5bfbab	Merge specaug change from Mingshuang.	2022-02-08 19:42:23 +08:00
Daniel Povey	395065eb11	Merge branch 'spec-augment-change' of https://github.com/luomingshuang/icefall into attention_relu_specaug	2022-02-08 19:40:33 +08:00
Mingshuang Luo	3323cabf46	Experiments based on SpecAugment change	2022-02-08 14:25:31 +08:00
Daniel Povey	a859dcb205	Remove learnable offset, use relu instead.	2022-02-07 12:14:48 +08:00
Wei Kang	35ecd7e562	Fix torch.nn.Embedding error for torch below 1.8.0 (#198)	2022-02-06 21:59:54 +08:00