mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Change activation in bottleneck to Tanh.

Browse Source
This commit is contained in:
Daniel Povey 2022-11-22 17:32:02 +08:00
parent 8dfeaa5f92
commit 19683aa516
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
View File

@ -1306,7 +1306,7 @@ class AttentionSqueeze(nn.Module):
max_factor=0.02,
min_prob=0.1,
)
self.bottleneck_activation = DoubleSwish() # in bottleneck
self.bottleneck_activation = nn.Tanh() # in bottleneck
self.activation = Identity() # for diagnostics
# the next two balancers are only to stop parameter-magnitude 'drift': we have