Are lm.binary and trie necessary for running evaluate.py

Hi Team,
So I read a lot of discussion about whether or not to provide lm.binary and trie while training, and it became quite clear that these two are needed only during the inference time.
However, what if I want to see the model prediction without lm.binary and trie. I read somewhere in dscourse that we can opt to not give the paths of lm.binary and trie but in that case it picks them up from their default location data/lm/
I tried removing that folder or re-naming it but then it threw up this error that lm/binary path must exist and be readable.
Another thing that I did was to provide lm_alpha=0 and lm_beta=0. Will it totally de-activate the use of language model while inference.
I was doing this concept testing around small corpus just to verify the working of lm.binary and trie.

Kindly help :slight_smile:

The only and correct way in the python code is to set scorer=None as you did.

Thanks a ton!! :slight_smile:

So @reuben did change that ealier today, not passing a LM path should now be enough.

1 Like

Thank you for the update:slight_smile:

One small detail: because the flags have default values, you have to explicitly pass an empty string for it to disable LM scoring. So --lm_binary_path '' --lm_trie_path ''.

– reuben

2 Likes

Will take that care! Thanks again :slightly_smiling_face: