wgb14
4316ec43d7
small fix
2021-12-03 16:34:36 -05:00
wgb14
64bd3f7df4
set audio duration mismatch tolerance to 0.01
2021-12-01 17:49:46 -05:00
Wang, Guanbo
b8beb00ecc
Merge pull request #2 from csukuangfj/fix-giga
...
Split manifests into 2000 pieces.
2021-11-30 00:28:58 -05:00
Fangjun Kuang
8109c2b913
Split manifests into 2000 pieces.
2021-11-30 12:04:15 +08:00
Wang, Guanbo
ee7c56c7d9
Merge pull request #1 from csukuangfj/fix-giga
...
Compute features for GigaSpeech by splitting the manifest.
2021-11-28 02:19:57 -05:00
Fangjun Kuang
4351e1ea14
Fixes after review.
2021-11-28 15:10:55 +08:00
Fangjun Kuang
317f5ec64e
Compute features for GigaSpeech by splitting the manifest.
2021-11-28 13:24:05 +08:00
wgb14
fa734e01a3
chunked feature extraction by default
2021-11-16 20:23:12 -05:00
wgb14
89c0e2e7ff
small fix
2021-11-14 01:13:07 -05:00
wgb14
9d08b44b19
small fix
2021-11-14 00:44:30 -05:00
wgb14
16f1799ef3
support HLG for BPE
2021-11-13 23:59:50 -05:00
wgb14
3dbb15bda2
support BPE based lang
2021-11-13 23:27:45 -05:00
wgb14
1d58765bd5
on-the-fly feature extraction by default
2021-11-13 17:45:35 -05:00
wgb14
75860159a2
support download, data prep, and fbank
2021-11-12 14:43:19 -05:00
wgb14
b7bda9eaf6
initial commit
2021-11-09 01:12:21 -05:00
Fangjun Kuang
04029871b6
Fix a bug in Nbest.compute_am_scores and Nbest.compute_lm_scores. ( #111 )
2021-11-09 13:44:51 +08:00
Fangjun Kuang
91cfecebf2
Remove duplicated token seq in rescoring. ( #108 )
...
* Remove duplicated token seq in rescoring.
* Use a larger range for ngram_lm_scale and attention_scale
2021-11-06 08:54:45 +08:00
Fangjun Kuang
810b193dcc
Clarify the doc about ctc-decoding. ( #104 )
2021-11-03 07:16:49 +08:00
Fangjun Kuang
42b437bea6
Use pre-sorted text to generate token ids for attention decoder. ( #98 )
...
* Use pre-sorted text to generate token ids for attention decoder.
See https://github.com/k2-fsa/icefall/issues/97
for more details.
* Fix typos.
2021-10-29 13:46:41 +08:00
Fangjun Kuang
12d647d899
Add a note about the CUDA OOM error. ( #94 )
...
* Add a note about the CUDA OOM error.
Some users consider this kind of OOM as an error during decoding,
but actually it is not. This pull request clarifies that.
* Fix style issues.
2021-10-29 12:17:56 +08:00
Fangjun Kuang
8cb7f712e4
Use GPU for averaging checkpoints if possible. ( #84 )
2021-10-26 17:10:04 +08:00
Fangjun Kuang
712ead8207
Fix an error when attention decoder rescoring returns None. ( #90 )
2021-10-22 19:52:25 +08:00
Piotr Żelasko
902e0b238d
Merge pull request #82 from pzelasko/feature/find-pessimistic-batches
...
Find CUDA OOM batches before starting training
2021-10-19 11:26:13 -04:00
Piotr Żelasko
3cc99d2af2
make flake8 happy
2021-10-19 11:24:54 -04:00
cdxie
d30244e28f
add a docker file for some users ( #87 )
...
* add a docker file for some users
Ubuntu18.04-pytorch1.7.1-cuda11.0-cudnn8-python3.8
* add a describing file of how to use dockerfile
give some steps to use dockerfile
2021-10-19 13:00:59 +08:00
Piotr Żelasko
86f3e0ef37
Make flake8 happy
2021-10-18 09:54:40 -04:00
Piotr Żelasko
6fbd7a287c
Refactor OOM batch scanning into a local function
2021-10-18 09:53:04 -04:00
Piotr Żelasko
d509d58f30
Merge branch 'master' into feature/find-pessimistic-batches
2021-10-18 09:47:21 -04:00
Fangjun Kuang
3effcb4225
Fix typos. ( #85 )
2021-10-18 16:17:14 +08:00
Fangjun Kuang
53b79fafa7
Add MMI training with word pieces as modelling unit. ( #6 )
...
* Fix an error in TDNN-LSTM training.
* WIP: Refactoring
* Refactor transformer.py
* Remove unused code.
* Minor fixes.
* Fix decoder padding mask.
* Add MMI training with word pieces.
* Remove unused files.
* Minor fixes.
* Refactoring.
* Minor fixes.
* Use pre-computed alignments in LF-MMI training.
* Minor fixes.
* Update decoding script.
* Add doc about how to check and use extracted alignments.
* Fix style issues.
* Fix typos.
* Fix style issues.
* Disable macOS tests for now.
2021-10-18 15:20:32 +08:00
Fangjun Kuang
4890e27b45
Extract framewise alignment information using CTC decoding ( #39 )
...
* Use new APIs with k2.RaggedTensor
* Fix style issues.
* Update the installation doc, saying it requires at least k2 v1.7
* Extract framewise alignment information using CTC decoding.
* Print environment information.
Print information about k2, lhotse, PyTorch, and icefall.
* Fix CI.
* Fix CI.
* Compute framewise alignment information of the LibriSpeech dataset.
* Update comments for the time to compute alignments of train-960.
* Preserve cut id in mix cut transformer.
* Minor fixes.
* Add doc about how to extract framewise alignments.
2021-10-18 14:24:33 +08:00
Jan "yenda" Trmal
bd7c2f7645
fix conformer typo in docs ( #83 )
2021-10-16 07:46:17 +08:00
Piotr Żelasko
403d1744ff
Introduce backprop in finding OOM batches
2021-10-15 10:05:13 -04:00
Piotr Żelasko
060117a9ff
Reformatting
2021-10-14 21:40:14 -04:00
Piotr Żelasko
1c7c79f2fc
Find CUDA OOM batches before starting training
2021-10-14 21:28:11 -04:00
Fangjun Kuang
fee1f84b20
Test pre-trained model in CI ( #80 )
...
* Add CI to run pre-trained models.
* Minor fixes.
* Install kaldifeat
* Install a CPU version of PyTorch.
* Fix CI errors.
* Disable decoder layers in pretrained.py if it is not used.
* Clone pre-trained model from GitHub.
* Minor fixes.
* Minor fixes.
* Minor fixes.
2021-10-15 00:41:33 +08:00
Mingshuang Luo
5401ce199d
Update ctc-decoding on pretrained.py and conformer_ctc.rst ( #78 )
2021-10-14 23:29:06 +08:00
Fangjun Kuang
f2387fe523
Fix a bug introduced while supporting torch script. ( #79 )
2021-10-14 20:09:38 +08:00
Fangjun Kuang
5016ee3c95
Give an informative message when users provide an unsupported decoding method ( #77 )
2021-10-14 16:20:35 +08:00
Mingshuang Luo
39bc8cae94
Add ctc decoding to pretrained.py on conformer_ctc ( #75 )
...
* Add ctc-decoding to pretrained.py
* update pretrained.py and conformer_ctc.rst
* update ctc-decoding for pretrained.py on conformer_ctc
* Update pretrained.py
* fix the style issue
* Update conformer_ctc.rst
* Update the running logs
2021-10-13 12:20:16 +08:00
Mingshuang Luo
391432b356
Update train.py ("10"--->"params.log_interval") ( #76 )
...
* Update train.py
* Update train.py
* Update train.py
2021-10-12 21:30:31 +08:00
Mingshuang Luo
597c5efdb1
Use LossRecord to record and print the loss for the training process ( #62 )
...
* Update index.rst (AS->ASR)
* Update conformer_ctc.rst (pretraind->pretrained)
* Fix some spelling errors.
* Fix some spelling errors.
* Use LossRecord to record and print loss in the training process
* Change the name "LossRecord" to "MetricsTracker"
2021-10-12 15:58:03 +08:00
Fangjun Kuang
beb54ddb61
Support torch script. ( #65 )
...
* WIP: Support torchscript.
* Minor fixes.
* Fix style issues.
* Add documentation about how to deploy a trained model.
2021-10-12 14:55:05 +08:00
Piotr Żelasko
d54828e73a
Merge pull request #73 from pzelasko/feature/bucketing-in-test
...
Use BucketingSampler for dev and test data
2021-10-09 10:58:29 -04:00
Piotr Żelasko
069ebaf9ba
Reformatting
2021-10-09 14:45:46 +00:00
Mingshuang Luo
6e43905d12
Update the documentation to include "ctc-decoding" ( #71 )
...
* Update conformer_ctc.rst
2021-10-09 11:56:25 +08:00
Piotr Żelasko
b682467e4d
Use BucketingSampler for dev and test data
2021-10-08 22:32:13 -04:00
Piotr Żelasko
adb068eb82
setup.py ( #64 )
2021-10-01 16:43:08 +08:00
Fangjun Kuang
707d7017a7
Support pure ctc decoding requiring neither a lexicon nor an n-gram LM ( #58 )
...
* Rename lattice_score_scale to nbest_scale.
* Support pure CTC decoding requiring neither a lexicion nor an n-gram LM.
* Fix style issues.
* Fix a typo.
* Minor fixes.
2021-09-26 14:21:49 +08:00
Fangjun Kuang
455693aede
Fix hasattr
of AttributeDict. ( #52 )
2021-09-22 16:37:20 +08:00