Daniel Povey
24aea947d2
Fix issues where grad is None, and unused-grad cases
2022-09-22 19:18:16 +08:00
Daniel Povey
c16f795962
Avoid error in ddp by using last module'sc scores
2022-09-22 18:52:16 +08:00
Daniel Povey
0f85a3c2e5
Implement persistent attention scores
2022-09-22 18:47:16 +08:00
Daniel Povey
03a77f8ae5
Merge branch 'scaled_adam_exp7c' into scaled_adam_exp11c
2022-09-22 18:15:44 +08:00
Daniel Povey
ceadfad48d
Reduce debug freq
2022-09-22 12:30:49 +08:00
Daniel Povey
1d20c12bc0
Increase max_var_per_eig to 0.2
2022-09-22 12:28:35 +08:00
Fangjun Kuang
9ae2f3a3c5
Small fixes to the transducer training doc ( #575 )
2022-09-21 14:20:49 +08:00
Fangjun Kuang
099cd3a215
support exporting to ncnn format via PNNX ( #571 )
2022-09-20 22:52:49 +08:00
Daniel Povey
e2fdfe990c
Loosen limit on param_max_rms, from 2.0 to 3.0; change how param_min_rms is applied.
2022-09-20 15:20:43 +08:00
Daniel Povey
6eb9a0bc9b
Halve max_var_per_eig to 0.05
2022-09-20 14:39:17 +08:00
Daniel Povey
cd5ac76a05
Add max-var-per-eig in encoder layers
2022-09-20 14:22:07 +08:00
Daniel Povey
db1f4ccdd1
4x scale on max-eig constraint
2022-09-20 14:20:13 +08:00
Teo Wen Shen
436942211c
Adding Dockerfile for Ubuntu18.04-pytorch1.12.1-cuda11.3-cudnn8 ( #572 )
...
* Changed Dockerfile
* Update Dockerfile
* Dockerfile
* Update README.md
* Add Dockerfiles
* Update README.md
Removed misleading CUDA version, as the Ubuntu18.04-pytorch1.7.1-cuda11.0-cudnn8 Dockerfile can only support CUDA versions >11.0.
2022-09-20 10:52:24 +08:00
Daniel Povey
3d72a65de8
Implement max-eig-proportion..
2022-09-19 10:26:37 +08:00
Daniel Povey
5f27cbdb44
Merge branch 'scaled_adam_exp4_max_var_per_eig' into scaled_adam_exp7
...
# Conflicts:
# egs/librispeech/ASR/pruned_transducer_stateless7/conformer.py
2022-09-18 21:23:59 +08:00
Daniel Povey
0f567e27a5
Add max_var_per_eig in self-attn
2022-09-18 21:22:01 +08:00
Daniel Povey
eb77fa7aaa
Restore min_positive,max_positive limits on linear_pos projection
2022-09-18 14:38:30 +08:00
Daniel Povey
69404f61ef
Use scalar_lr_scale for scalars as well as sizes.
2022-09-18 14:12:27 +08:00
Daniel Povey
76031a7c1d
Loosen some limits of activation balancers
2022-09-18 13:59:44 +08:00
Daniel Povey
3122637266
Use ScaledLinear where I previously had StructuredLinear
2022-09-17 13:18:58 +08:00
Daniel Povey
4a2b940321
Remove StructuredLinear,StructuredConv1d
2022-09-17 13:14:08 +08:00
Daniel Povey
1a184596b6
A little code refactoring
2022-09-16 20:56:21 +08:00
Fangjun Kuang
97b3fc53aa
Add LSTM for the multi-dataset setup. ( #558 )
...
* Add LSTM for the multi-dataset setup.
* Add results
* fix style issues
* add missing file
2022-09-16 18:40:25 +08:00
Daniel Povey
bb1bee4a7b
Improve how quartiles are printed
2022-09-16 17:30:03 +08:00
Daniel Povey
5f55f80fbb
Configure train.py with clipping_scale=2.0
2022-09-16 17:19:52 +08:00
Daniel Povey
8298333bd2
Implement gradient clipping.
2022-09-16 16:52:46 +08:00
Daniel Povey
8f876b3f54
Remove batching from ScaledAdam, in preparation to add gradient norm clipping
2022-09-16 15:42:56 +08:00
Daniel Povey
3b450c2682
Bug fix in train.py, fix optimzier name
2022-09-16 14:10:42 +08:00
Daniel Povey
257c961b66
1st attempt at scaled_adam
2022-09-16 13:59:52 +08:00
Fangjun Kuang
145c44f710
Use modified ctc topo when vocab size is > 500 ( #568 )
2022-09-13 10:59:27 +08:00
shcxlee
9e24642faf
Modified prepare_transcripts.py and preprare_lexicon.py of tedlium3 recipe ( #567 )
2022-09-10 10:32:49 +08:00
Fangjun Kuang
e18fa78c3a
Check that read_manifests_if_cached returns a non-empty dict. ( #555 )
2022-08-28 11:50:11 +08:00
Fangjun Kuang
d68b8e9120
Disable CUDA_LAUNCH_BLOCKING in wenetspeech recipes. ( #554 )
...
* Disable CUDA_LAUNCH_BLOCKING in wenetspeech recipes.
* minor fixes
2022-08-28 11:17:38 +08:00
kobenaxie
235eb0746f
fix scaling converter test for decoder(predictor). ( #553 )
2022-08-27 17:26:21 +08:00
rickychanhoyin
2636a3dd58
minor changes for correct path names && import module text2segments.py ( #552 )
...
* Update asr_datamodule.py
minor file names correction
* minor changes for correct path names && import module text2segments.py
2022-08-27 17:23:45 +08:00
marcoyang1998
1e31fbcd7d
Add clamping operation in Eve optimizer for all scalar weights to avoid ( #550 )
...
non stable training in some scenarios. The clamping range is set to (-10,2).
Note that this change may cause unexpected effect if you resume
training from a model that is trained without clamping.
2022-08-25 12:12:50 +08:00
Duo Ma
0967cf5b38
fixed no cut_id error in decode_dataset ( #549 )
...
* fixed import quantization is none
Signed-off-by: shanguanma <nanr9544@gmail.com>
* fixed no cut_id error in decode_dataset
Signed-off-by: shanguanma <nanr9544@gmail.com>
* fixed more than one "#"
Signed-off-by: shanguanma <nanr9544@gmail.com>
* fixed code style
Signed-off-by: shanguanma <nanr9544@gmail.com>
Signed-off-by: shanguanma <nanr9544@gmail.com>
Co-authored-by: shanguanma <nanr9544@gmail.com>
2022-08-25 10:54:21 +08:00
Daniel Povey
276928655e
Merge branch 'pradam_exp1m8' into pradam_exp1m7s2
2022-08-24 04:17:30 +08:00
Daniel Povey
80beb9c8d7
Merge branch 'pradam_exp1n2' into pradam_exp1m7s2
2022-08-24 04:14:25 +08:00
rxhmdia
626a26fc2a
some small changes for aidatatang_200zh ( #542 )
...
* Update prepare.sh
* Update compute_fbank_aidatatang_200zh.py
2022-08-23 17:30:03 +08:00
Yuekai Zhang
f9c3d7f92f
fix typo for export jit script ( #544 )
2022-08-23 17:29:42 +08:00
Duo Ma
dbd61a9db3
fixed import quantization is none ( #541 )
...
Signed-off-by: shanguanma <nanr9544@gmail.com>
Signed-off-by: shanguanma <nanr9544@gmail.com>
Co-authored-by: shanguanma <nanr9544@gmail.com>
2022-08-23 10:19:03 +08:00
Zengwei Yao
c0101185d7
consider case of empty tensor ( #540 )
2022-08-22 21:42:56 +08:00
Lucky Wong
9277c95bcd
Pruned transducer stateless2 for AISHELL-1 ( #536 )
...
* Fix not enough values to unpack error .
* [WIP] Pruned transducer stateless2 for AISHELL-1
* fix the style issue
* code format for black
* add pruned-transducer-stateless2 results for AISHELL-1
* simplify result
2022-08-22 10:17:26 +08:00
Fangjun Kuang
0598291ff1
minor fixes to LSTM streaming model ( #537 )
2022-08-20 09:50:50 +08:00
rickychanhoyin
cdea2d26d4
Update asr_datamodule.py ( #538 )
...
minor file names correction
2022-08-20 00:16:38 +08:00
Zengwei Yao
f2f5baf687
Use ScaledLSTM as streaming encoder ( #479 )
...
* add ScaledLSTM
* add RNNEncoderLayer and RNNEncoder classes in lstm.py
* add RNN and Conv2dSubsampling classes in lstm.py
* hardcode bidirectional=False
* link from pruned_transducer_stateless2
* link scaling.py pruned_transducer_stateless2
* copy from pruned_transducer_stateless2
* modify decode.py pretrained.py test_model.py train.py
* copy streaming decoding files from pruned_transducer_stateless2
* modify streaming decoding files
* simplified code in ScaledLSTM
* flat weights after scaling
* pruned2 -> pruned4
* link __init__.py
* fix style
* remove add_model_arguments
* modify .flake8
* fix style
* fix scale value in scaling.py
* add random combiner for training deeper model
* add using proj_size
* add scaling converter for ScaledLSTM
* support jit trace
* add using averaged model in export.py
* modify test_model.py, test if the model can be successfully exported by jit.trace
* modify pretrained.py
* support streaming decoding
* fix model.py
* Add cut_id to recognition results
* Add cut_id to recognition results
* do not pad in Conv subsampling module; add tail padding during decoding.
* update RESULTS.md
* minor fix
* fix doc
* update README.md
* minor change, filter infinite loss
* remove the condition of raise error
* modify type hint for the return value in model.py
* minor change
* modify RESULTS.md
Co-authored-by: pkufool <wkang.pku@gmail.com>
2022-08-19 14:38:45 +08:00
Lucky Wong
31686ac829
Fix not enough values to unpack error . ( #533 )
2022-08-18 10:45:06 +08:00
Daniel Povey
64f7166545
Some cleanups
2022-08-18 07:03:50 +08:00
marcoyang1998
c74cec59e9
propagate changes from #525 to other librispeech recipes ( #531 )
...
* propaga changes from #525 to other librispeech recipes
* refactor display_and_save_batch to utils
* fixed typo
* reformat code style
2022-08-17 17:18:15 +08:00
