1615 Commits

Author	SHA1	Message	Date
Daniel Povey	b656b0df36	Changes that should affect nothing: bug fixes etc.	2023-06-18 04:00:43 +08:00
Daniel Povey	cdd9cf695f	Fix bug regarding --start-batch option	2023-05-29 16:41:54 +08:00
Daniel Povey	cbd59b9c68	Don't skip penalize_abs_values_gt due to memory cutoff; remove grad_scale=0.1	2023-05-29 16:29:48 +08:00
Daniel Povey	79f1863a1e	Fix SoftmaxFunction bug	2023-05-29 10:55:03 +08:00
Daniel Povey	137ac513bf	Some changes to try to reduce mem consumption; decrease batch size	2023-05-28 21:50:34 +08:00
Daniel Povey	625e39fd1a	Avoid penalize_abs_values_gt when memory usage high	2023-05-28 20:40:47 +08:00
Daniel Povey	bcc9971ebe	Add clip_grad	2023-05-23 14:00:56 +08:00
Daniel Povey	cdfa388ac0	Revert optim schedule	2023-05-18 15:35:23 +08:00
Daniel Povey	e4a774cb98	Warm up lr more slowly	2023-05-18 15:03:44 +08:00
Daniel Povey	bfeeddda81	Reduce mem consumption of softmax backward	2023-05-16 12:18:09 +08:00
Daniel Povey	671e9ee5bd	Restore old warmup schedule	2023-05-15 20:40:41 +08:00
Daniel Povey	2e66392306	Change warmup schedule	2023-05-15 20:20:15 +08:00
Daniel Povey	86c2c60100	Step lr_scheduler on tokens not epoch; add some more debug output	2023-05-04 15:35:22 +08:00
Daniel Povey	3574e7dbb5	Initial version of zipformer1 LM that runs, not sure whether it is working	2023-05-04 14:46:06 +08:00
Daniel Povey	75e9f1a34a	Fix bug with indicator	2023-05-02 13:36:03 +08:00
Daniel Povey	c207c55e94	alias Transducer	2023-05-02 13:19:21 +08:00
Daniel Povey	1ab2a4c662	Add text embeddings, but use actual text for now	2023-05-01 22:09:27 +08:00
Daniel Povey	fa696e919b	Add memory to model	2023-05-01 20:47:09 +08:00
Daniel Povey	6f5c4688ef	Add (back) straight_through_rate, with rate 0.025; try to handle memory allocation failures in backprop better.	2023-04-30 15:19:34 +08:00
Daniel Povey	e4626a14b8	Change length_factor from 3.0 to 1.0	2023-04-27 22:38:45 +08:00
Daniel Povey	6c26754628	Fix tests, make SwooshL and SwooshR more efficient in forward pass.	2023-04-27 22:37:19 +08:00
yaozengwei	55a1abc9da	separate Conv2dSubsampling from Zipformer	2023-04-27 10:11:47 +08:00
yaozengwei	0ec31c84da	remove skip_modules	2023-04-24 15:50:12 +08:00
yaozengwei	2e80841790	set --lr-batches=7500	2023-04-24 15:49:41 +08:00
yaozengwei	9291a39f58	remove all lr_scales, set layer3_channels=128, change the position of feed_forward1	2023-04-24 15:45:38 +08:00
yaozengwei	2cd1933873	remove similar-named args in decode.py	2023-04-14 14:24:57 +08:00
yaozengwei	87d9491fba	minor fix in decode.py, about args	2023-04-13 17:20:25 +08:00
yaozengwei	d27e61170b	set --base-lr=0.045 as default	2023-04-12 19:12:07 +08:00
yaozengwei	0b0732ae28	use ActivationDropoutAndLinearFunction and swoosh kernel functions	2023-04-12 19:11:26 +08:00
Daniel Povey	73099da6be	Changes to frame masking: done at sequence level, with 2 dimension cutoffs	2023-04-10 15:30:56 +08:00
Daniel Povey	fb6a1c1464	Replace SimpleCombiner with BypassModule, for simplicity ... Refactor code for simplicity Fix bug	2023-04-10 14:57:54 +08:00
Daniel Povey	5f790c41f7	Changes to bypass layer-skip-rate configuration.	2023-04-06 23:55:25 +08:00
Daniel Povey	67fcae95a8	Refactor bypass, and add bypass in the middle of the layer.	2023-04-05 14:44:16 +08:00
Daniel Povey	b526f3af00	Increase num layers	2023-04-04 15:39:32 +08:00
Daniel Povey	c4f669ef00	Increase feedforward dims and num layers	2023-04-04 14:41:23 +08:00
Daniel Povey	7ab1e7f5ec	Combine two layers into one.	2023-04-04 12:14:18 +08:00
Daniel Povey	3dd25d6b2d	Increase feature_mask_dropout_prob to 0.125	2023-04-03 12:13:09 +08:00
Daniel Povey	c2e39bd488	Bug fix	2023-03-31 17:23:25 +08:00
Daniel Povey	cd0f48f508	Mask larger regions	2023-03-31 17:07:22 +08:00
Daniel Povey	d41b73000e	Modify feature_mask_dropout_prob	2023-03-31 13:25:39 +08:00
Daniel Povey	e64ec396bd	Have 2 not 3 groups, but give 1st group a smaller dropout prob than the 2nd.	2023-03-30 16:38:41 +08:00
Daniel Povey	6e058b9ebd	Fix or vs. and bug	2023-03-30 00:00:59 +08:00
Daniel Povey	a02199df79	Fix bug	2023-03-29 20:33:43 +08:00
Daniel Povey	f1dbf4222e	Divide feature_mask into 3 groups	2023-03-29 16:22:39 +08:00
Daniel Povey	b8f0756133	Add comment	2023-03-29 14:05:28 +08:00
Daniel Povey	bb8cbd7598	Sometimes mask more frames.	2023-03-29 13:08:52 +08:00
Daniel Povey	4e36656cef	Remove import that is no longer there	2023-03-10 14:45:02 +08:00
Daniel Povey	07b685936a	Fix typo	2023-03-07 21:53:12 +08:00
Daniel Povey	e692e0b228	Add balancer for keys	2023-03-07 17:39:01 +08:00
Daniel Povey	f59da65d82	Remove some more unused code; rename BasicNorm->BiasNorm, Zipformer->Zipformer2	2023-03-06 14:27:11 +08:00