Increase power from 0.7 to 0.75

This commit is contained in:
Daniel Povey 2022-07-19 05:44:03 +08:00
parent 2dff1161b4
commit 525c097130

View File

@ -142,7 +142,7 @@ param_rms_smooth1: Smoothing proportion for parameter matrix, if assumed rank of
lr=3e-02, lr=3e-02,
betas=(0.9, 0.98), betas=(0.9, 0.98),
size_lr_scale=0.1, size_lr_scale=0.1,
param_pow=0.7, param_pow=0.75,
param_rms_smooth0=0.75, param_rms_smooth0=0.75,
param_rms_smooth1=0.25, param_rms_smooth1=0.25,
max_lr_factor=2.5, max_lr_factor=2.5,