mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-08-08 09:32:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Initialize BiasNorm bias with small random values (#1630)

Browse Source
This commit is contained in:
Zengwei Yao 2024-05-20 22:32:02 +08:00 committed by GitHub
parent 68980c5d0a
commit 0df406c5da
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
egs/librispeech/ASR/zipformer/scaling.py
View File

@ -462,7 +462,7 @@ class BiasNorm(torch.nn.Module):
self.num_channels = num_channels
self.channel_dim = channel_dim
self.log_scale = nn.Parameter(torch.tensor(log_scale))
self.bias = nn.Parameter(torch.zeros(num_channels))
self.bias = nn.Parameter(torch.empty(num_channels).normal_(mean=0, std=1e-4))
self.log_scale_min = log_scale_min
self.log_scale_max = log_scale_max