marcoyang1998 0d1cd4f595 add char coverage option to avoid having a lot of rarely used tokens in the BPE; add the option to use byte-fallback in training BPE

2023-07-19 10:55:57 +08:00

..

bpe2tokens.py

minor fixes

2023-07-18 11:14:06 +08:00

compute_fbank_libriheavy.py

minor fixes

2023-07-18 11:14:06 +08:00

filter_cuts.py

add necessary files to compute features

2023-07-17 10:36:25 +08:00

prepare_lang_bpe.py

add necessary files to compute features

2023-07-17 10:36:25 +08:00

prepare_lang.py

add necessary utility files

2023-07-18 10:06:22 +08:00

train_bpe_model.py

add char coverage option to avoid having a lot of rarely used tokens in the BPE; add the option to use byte-fallback in training BPE

2023-07-19 10:55:57 +08:00