7 Commits

Author	SHA1	Message	Date
marcoyang1998	0d1cd4f595	add char coverage option to avoid having a lot of rarely used tokens in the BPE; add the option to use byte-fallback in training BPE	2023-07-19 10:55:57 +08:00
marcoyang1998	b53c0d1e5f	initial commit for zipformer recipe	2023-07-18 11:42:19 +08:00
marcoyang1998	6939b3d6aa	minor fixes	2023-07-18 11:14:06 +08:00
marcoyang	0e7df7c5c4	add necessary utility files	2023-07-18 10:06:22 +08:00
marcoyang	189d424b25	only use medium text to train the BPE as the whole corpus is tooooo large	2023-07-18 10:06:01 +08:00
marcoyang	fef229e024	add necessary files to compute features	2023-07-17 10:36:25 +08:00
marcoyang	44d01195c0	initial commit for libriheavy	2023-07-14 23:50:27 +08:00