mirrors/icefall
1
0
Fork 0
mirror of https://github.com/k2-fsa/icefall.git synced 2025-08-09 18:12:19 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update make_kn_lm.py

Browse Source
This commit is contained in:
huangruizhe 2022-01-02 00:27:27 -08:00 committed by GitHub
parent 49aab7e658
commit 0a67015d63
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
icefall/shared/make_kn_lm.py
View File

@ -165,9 +165,9 @@ class NgramCounts:
n1 += stat[1]
n2 += stat[2]
assert n1 + 2 * n2 > 0
self.d.append(max(0.001, n1 * 1.0) / (n1 + 2 * n2)) # We are doing this max(0.001, xxx) to avoid zero discounting constant D,
# which could happen if the number of symbols is small and all w in the vocab
# has been seen after certain h. This can cause division by zero in computing BOW.
self.d.append(max(0.001, n1 * 1.0) / (n1 + 2 * n2)) # We are doing this max(0.001, xxx) to avoid zero discounting constant D due to n1=0,
# which could happen if the number of symbols is small.
# Otherwise, zero discounting constant can cause division by zero in computing BOW.
def cal_f(self):
# f(a_z) is a probability distribution of word sequence a_z.