mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-12-11 06:55:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

Invoke the out_balancer of attention_squeeze

Browse Source
This commit is contained in:
Daniel Povey 2022-12-12 23:31:22 +08:00
parent 0522425ea8
commit b5e0676f14
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
egs/librispeech/ASR/pruned_transducer_stateless7/zipformer.py
View File

@ -1445,6 +1445,7 @@ attn_weights: a Tensor of shape (num_heads, batch_size, seq_len, seq_len)
x = x * scales x = x * scales
x = self.activation(x) # Identity only. For diagnostics. x = self.activation(x) # Identity only. For diagnostics.
x = self.out_proj(x) x = self.out_proj(x)
x = self.out_balancer(x)
return x return x