mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Make 20 the limit for warmup_count

Browse Source
This commit is contained in:
Daniel Povey 2022-10-25 12:58:27 +08:00
parent 6ebff23cb9
commit 3159b09e8f
1 changed files with 5 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
egs/librispeech/ASR/pruned_transducer_stateless7/conformer.py
View File

@ -450,11 +450,14 @@ class ConformerEncoder(nn.Module):
initial_layerdrop_prob = 0.5 initial_layerdrop_prob = 0.5
final_layerdrop_prob = 0.05 final_layerdrop_prob = 0.05
if warmup_count < 5.0: if warmup_count < 20.0:
# As a special case, if warmup_count < 5.0 return 0 (drop no # As a special case, if warmup_count < 20.0 return 0 (drop no
# layers). This is rather ugly, I'm afraid; it is intended to # layers). This is rather ugly, I'm afraid; it is intended to
# enable our scan_pessimistic_batches_for_oom() code to work correctly # enable our scan_pessimistic_batches_for_oom() code to work correctly
# so if we are going to get OOM it will happen early. # so if we are going to get OOM it will happen early.
# also search for 'warmup_count' with quotes in this file to see
# how we initialize the warmup count to a random number between
# 0 and 10.
return 0.0 return 0.0
elif warmup_count < layer_warmup_begin: elif warmup_count < layer_warmup_begin:
return initial_layerdrop_prob return initial_layerdrop_prob