Change max rms from 10.0 to 4.0

This commit is contained in:
Daniel Povey 2022-07-13 05:53:35 +08:00
parent cedfb5a377
commit d48fe0b99c

View File

@ -962,7 +962,7 @@ param_rms_smooth1: Smoothing proportion for parameter matrix, if assumed rank of
if True:
# Apply max_rms
max_rms = 10.0
max_rms = 4.0
ans.clamp_(max=max_rms*2)
ans /= _mean(ans, exclude_dims=[0], keepdim=True)
ans.clamp_(max=max_rms)