mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Remove max_factor=0.02 option in bottleneck balancer of class AttentionSqueeze, change its min,max positive to 0.2,0.8

Browse Source
This commit is contained in:
Daniel Povey 2022-11-29 15:43:25 +08:00
parent 5632782ee1
commit 28c5923986
1 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
View File

@ -1327,10 +1327,9 @@ class AttentionSqueeze(nn.Module):
# beginning, and make it looser over time. # beginning, and make it looser over time.
self.bottleneck_balancer = ActivationBalancer( self.bottleneck_balancer = ActivationBalancer(
bottleneck_dim, channel_dim=-1, bottleneck_dim, channel_dim=-1,
min_positive=0.05, max_positive=0.95, min_positive=0.2, max_positive=0.8,
min_abs=0.05, min_abs=0.05,
max_abs=ScheduledFloat((0.0, 0.5), (4000.0, 1.0), default=1.0), max_abs=ScheduledFloat((0.0, 0.5), (4000.0, 1.0), default=1.0),
max_factor=0.02,
min_prob=0.1, min_prob=0.1,
) )
self.bottleneck_activation = TanSwish() # in bottleneck self.bottleneck_activation = TanSwish() # in bottleneck