54 Commits

Author SHA1 Message Date
Fangjun Kuang
1d44da845b
RNN-T Conformer training for LibriSpeech (#143)
* Begin to add RNN-T training for librispeech.

* Copy files from conformer_ctc.

Will edit it.

* Use conformer/transformer model as encoder.

* Begin to add training script.

* Add training code.

* Remove long utterances to avoid OOM when a large max_duraiton is used.

* Begin to add decoding script.

* Add decoding script.

* Minor fixes.

* Add beam search.

* Use LSTM layers for the encoder.

Need more tunings.

* Use stateless decoder.

* Minor fixes to make it ready for merge.

* Fix README.

* Update RESULT.md to include RNN-T Conformer.

* Minor fixes.

* Fix tests.

* Minor fixes.

* Minor fixes.

* Fix tests.
2021-12-18 07:42:51 +08:00
Wei Kang
a183d5bfd7
Remove batchnorm (#147)
* Remove batch normalization

* Minor fixes

* Fix typo

* Fix comments

* Add assertion for use_feat_batchnorm
2021-12-14 08:20:03 +08:00
Fangjun Kuang
ec591698b0
Associate a cut with token alignment (without repeats) (#125)
* WIP: Associate a cut with token alignment (without repeats)

* Save framewise alignments with/without repeats.

* Minor fixes.
2021-11-29 18:50:54 +08:00
Fangjun Kuang
243fb9723c
Fix an error introduced while supporting torchscript. (#134)
Should be `G.dummy = 1`, not `G["dummy"] = 1`.
2021-11-27 09:07:04 +08:00
Fangjun Kuang
0e541f5b5d
Print hostname and IP address to the log. (#131)
We are using multiple machines to do various experiments. It makes
life easier to know which experiment is running on which machine
if we also log the IP and hostname of the machine.
2021-11-26 11:25:59 +08:00
Wei Kang
4151cca147
Add torch script support for Aishell and update documents (#124)
* Add aishell recipe

* Remove unnecessary code and update docs

* adapt to k2 v1.7, add docs and results

* Update conformer ctc model

* Update docs, pretrained.py & results

* Fix code style

* Fix code style

* Fix code style

* Minor fix

* Minor fix

* Fix pretrained.py

* Update pretrained model & corresponding docs

* Export torch script model for Aishell

* Add C++ deployment docs

* Minor fixes

* Fix unit test

* Update Readme
2021-11-19 16:37:05 +08:00
Fangjun Kuang
0660d12e4e
Fix computing WERs for empty hypotheses (#118)
* Fix computing WERs when empty lattices are generated.

* Minor fixes.
2021-11-17 19:25:47 +08:00
Fangjun Kuang
336283f872
New label smoothing (#109)
* Modify label smoothing to match the one implemented in PyTorch.

* Enable CI for torch 1.10

* Fix CI errors.

* Fix CI installation errors.

* Fix CI installation errors.

* Minor fixes.

* Minor fixes.

* Minor fixes.

* Minor fixes.

* Minor fixes.

* Fix CI errors.
2021-11-17 19:24:07 +08:00
Fangjun Kuang
68506609ad
Set fsa.properties to None after changing its labels in-place. (#121) 2021-11-16 23:11:30 +08:00
Fangjun Kuang
8d679c3e74
Fix typos. (#115) 2021-11-10 14:45:30 +08:00
Fangjun Kuang
21096e99d8
Update result for the librispeech recipe using vocab size 500 and att rate 0.8 (#113)
* Update RESULTS using vocab size 500, att rate 0.8

* Update README.

* Refactoring.

Since FSAs in an Nbest object are linear in structure, we can
add the scores of a path to compute the total scores.

* Update documentation.

* Change default vocab size from 5000 to 500.
2021-11-10 14:32:52 +08:00
Fangjun Kuang
42b437bea6
Use pre-sorted text to generate token ids for attention decoder. (#98)
* Use pre-sorted text to generate token ids for attention decoder.

See https://github.com/k2-fsa/icefall/issues/97
for more details.

* Fix typos.
2021-10-29 13:46:41 +08:00
Fangjun Kuang
8cb7f712e4
Use GPU for averaging checkpoints if possible. (#84) 2021-10-26 17:10:04 +08:00
Fangjun Kuang
712ead8207
Fix an error when attention decoder rescoring returns None. (#90) 2021-10-22 19:52:25 +08:00
Piotr Żelasko
3cc99d2af2 make flake8 happy 2021-10-19 11:24:54 -04:00
Piotr Żelasko
86f3e0ef37 Make flake8 happy 2021-10-18 09:54:40 -04:00
Piotr Żelasko
6fbd7a287c Refactor OOM batch scanning into a local function 2021-10-18 09:53:04 -04:00
Piotr Żelasko
d509d58f30 Merge branch 'master' into feature/find-pessimistic-batches 2021-10-18 09:47:21 -04:00
Fangjun Kuang
3effcb4225
Fix typos. (#85) 2021-10-18 16:17:14 +08:00
Fangjun Kuang
53b79fafa7
Add MMI training with word pieces as modelling unit. (#6)
* Fix an error in TDNN-LSTM training.

* WIP: Refactoring

* Refactor transformer.py

* Remove unused code.

* Minor fixes.

* Fix decoder padding mask.

* Add MMI training with word pieces.

* Remove unused files.

* Minor fixes.

* Refactoring.

* Minor fixes.

* Use pre-computed alignments in LF-MMI training.

* Minor fixes.

* Update decoding script.

* Add doc about how to check and use extracted alignments.

* Fix style issues.

* Fix typos.

* Fix style issues.

* Disable macOS tests for now.
2021-10-18 15:20:32 +08:00
Fangjun Kuang
4890e27b45
Extract framewise alignment information using CTC decoding (#39)
* Use new APIs with k2.RaggedTensor

* Fix style issues.

* Update the installation doc, saying it requires at least k2 v1.7

* Extract framewise alignment information using CTC decoding.

* Print environment information.

Print information about k2, lhotse, PyTorch, and icefall.

* Fix CI.

* Fix CI.

* Compute framewise alignment information of the LibriSpeech dataset.

* Update comments for the time to compute alignments of train-960.

* Preserve cut id in mix cut transformer.

* Minor fixes.

* Add doc about how to extract framewise alignments.
2021-10-18 14:24:33 +08:00
Piotr Żelasko
403d1744ff Introduce backprop in finding OOM batches 2021-10-15 10:05:13 -04:00
Piotr Żelasko
060117a9ff Reformatting 2021-10-14 21:40:14 -04:00
Piotr Żelasko
1c7c79f2fc Find CUDA OOM batches before starting training 2021-10-14 21:28:11 -04:00
Fangjun Kuang
fee1f84b20
Test pre-trained model in CI (#80)
* Add CI to run pre-trained models.

* Minor fixes.

* Install kaldifeat

* Install a CPU version of PyTorch.

* Fix CI errors.

* Disable decoder layers in pretrained.py if it is not used.

* Clone pre-trained model from GitHub.

* Minor fixes.

* Minor fixes.

* Minor fixes.
2021-10-15 00:41:33 +08:00
Mingshuang Luo
5401ce199d
Update ctc-decoding on pretrained.py and conformer_ctc.rst (#78) 2021-10-14 23:29:06 +08:00
Fangjun Kuang
f2387fe523
Fix a bug introduced while supporting torch script. (#79) 2021-10-14 20:09:38 +08:00
Fangjun Kuang
5016ee3c95
Give an informative message when users provide an unsupported decoding method (#77) 2021-10-14 16:20:35 +08:00
Mingshuang Luo
39bc8cae94
Add ctc decoding to pretrained.py on conformer_ctc (#75)
* Add ctc-decoding to pretrained.py

* update pretrained.py and conformer_ctc.rst

* update ctc-decoding for pretrained.py on conformer_ctc

* Update pretrained.py

* fix the style issue

* Update conformer_ctc.rst

* Update the running logs
2021-10-13 12:20:16 +08:00
Mingshuang Luo
391432b356
Update train.py ("10"--->"params.log_interval") (#76)
* Update train.py

* Update train.py

* Update train.py
2021-10-12 21:30:31 +08:00
Mingshuang Luo
597c5efdb1
Use LossRecord to record and print the loss for the training process (#62)
* Update index.rst (AS->ASR)

* Update conformer_ctc.rst (pretraind->pretrained)

* Fix some spelling errors.

* Fix some spelling errors.

* Use LossRecord to record and print loss in the training process

* Change the name "LossRecord" to "MetricsTracker"
2021-10-12 15:58:03 +08:00
Fangjun Kuang
beb54ddb61
Support torch script. (#65)
* WIP: Support torchscript.

* Minor fixes.

* Fix style issues.

* Add documentation about how to deploy a trained model.
2021-10-12 14:55:05 +08:00
Fangjun Kuang
707d7017a7
Support pure ctc decoding requiring neither a lexicon nor an n-gram LM (#58)
* Rename lattice_score_scale to nbest_scale.

* Support pure CTC decoding requiring neither a lexicion nor an n-gram LM.

* Fix style issues.

* Fix a typo.

* Minor fixes.
2021-09-26 14:21:49 +08:00
Fangjun Kuang
a80e58e15d
Refactor decode.py to make it more readable and more modular. (#44)
* Refactor decode.py to make it more readable and more modular.

* Fix an error.

Nbest.fsa should always have token IDs as labels and
word IDs as aux_labels.

* Add nbest decoding.

* Compute edit distance with k2.

* Refactor nbest-oracle.

* Add rescore with nbest lists.

* Add whole-lattice rescoring.

* Add rescoring with attention decoder.

* Refactoring.

* Fixes after refactoring.

* Fix a typo.

* Minor fixes.

* Replace [] with () for shapes.

* Use k2 v1.9

* Use Levenshtein graphs/alignment from k2 v1.9

* [doc] Require k2 >= v1.9

* Minor fixes.
2021-09-20 15:44:54 +08:00
Wei Kang
24656e9749
Update docs and remove unnecessary arguments (#42)
* Fix typo in docs

* Update docs and remove unnecessary arguments

* Fix code style
2021-09-13 18:28:57 +08:00
Fangjun Kuang
f792b466bf
Change default value of lattice-score-scale from 1.0 to 0.5 (#41)
* Change the default value of lattice-score-scale from 1.0 to 0.5

* Fix CI.
2021-09-13 10:49:18 +08:00
Fangjun Kuang
abadc71415
Use new APIs with k2.RaggedTensor (#38)
* Use new APIs with k2.RaggedTensor

* Fix style issues.

* Update the installation doc, saying it requires at least k2 v1.7

* Use k2 v1.7
2021-09-08 14:55:30 +08:00
Fangjun Kuang
184dbb3ea5
Add documentation about code style and creating new recipes. (#27) 2021-08-25 14:48:41 +08:00
pkufool
f4223ee110
Add TDNN-LSTM-CTC Results (#25)
* Add tdnn-lstm pretrained model and results

* Add docs for TDNN-LSTM-CTC

* Minor fix

* Fix typo

* Fix style checking
2021-08-24 21:09:27 +08:00
Fangjun Kuang
1bd5dcc8ac
WIP: Add doc for the LibriSpeech recipe. (#24)
* WIP: Add doc for the LibriSpeech recipe.

* Add more doc for LibriSpeech recipe.

* Add more doc for the LibriSpeech recipe.

* More doc.
2021-08-24 20:28:32 +08:00
pkufool
19c4214958
Fix code style and add copyright. (#18)
* Fix style and add copyright

* Minor fix

* Remove duplicate lines

* Reformat conformer.py by black

* Reformat code style with black.

* Fix github workflows

* Fix lhotse installation

* Install icefall requirements

* Update k2 version, remove lhotse from test workflow
2021-08-23 10:43:59 +08:00
Fangjun Kuang
8469f9ae0a
Refactor asr_datamodule. (#15)
* WIP: Refactor asr_datamodule.

* Fixes after review.

* Minor fixes.
2021-08-21 09:53:46 +08:00
Fangjun Kuang
0b656e4e1c
Add a link to Colab. (#14)
It demonstrates the usages of pre-trained models.
2021-08-20 15:43:25 +08:00
Fangjun Kuang
9d0cc9d829
Support computing nbest oracle WER. (#10)
* Support computing nbest oracle WER.

* Add scale to all nbest based decoding/rescoring methods.

* Add script to run pretrained models.

* Use torchaudio to extract features.

* Support decoding multiple files at the same time.

Also, use kaldifeat for feature extraction.

* Support decoding with LM rescoring and attention-decoder rescoring.

* Minor fixes.

* Replace scale with lattice-score-scale.

* Add usage example with a provided pretrained model.
2021-08-20 11:53:37 +08:00
pkufool
ef233486ae
The training script produce WER of 2.57% on librispeech test-clean (#13)
* Add grad_clip and weight-decay, small fix of dataloader and masking

* Add RESULTS.md
2021-08-20 10:08:08 +08:00
Fangjun Kuang
caa0b9e942
Fix an error in displaying decoding process. (#12) 2021-08-19 14:54:01 +08:00
Fangjun Kuang
12a2fd023e
Add doc about installation and usage (#7)
* Add readme.

* Add TOC.

* fix typos

* Minor fixes after review.
2021-08-12 12:44:04 +08:00
Fangjun Kuang
5a0b9bcb23
Refactoring (#4)
* Fix an error in TDNN-LSTM training.

* WIP: Refactoring

* Refactor transformer.py

* Remove unused code.

* Minor fixes.
2021-08-04 14:53:02 +08:00
Fangjun Kuang
398ed80d7a Minor fixes to support DDP training. 2021-07-31 15:26:57 +08:00
Fangjun Kuang
b94d97da37 Disable gradient computation in evaluation mode. 2021-07-29 20:37:31 +08:00