pythonnlpnltkmetrics

TypeError: "hypothesis" expects pre-tokenized hypothesis (Iterable[str]):


I am trying to calculate the Meteor score for the following:

print (nltk.translate.meteor_score.meteor_score(
    ["this is an apple", "that is an apple"], "an apple on this tree"))

However I am getting this error every time and I am not sure how to fix it.

TypeError: "hypothesis" expects pre-tokenized hypothesis (Iterable[str]): an apple on this tree

I also tried to put "an apple on this tree" in a list

    from nltk.translate.meteor_score import meteor_score
import nltk 
print (nltk.translate.meteor_score.meteor_score(
    ["this is an apple", "that is an apple"], ["an apple on this tree"]))

but it gave me this error.

TypeError: "reference" expects pre-tokenized reference (Iterable[str]): this is an apple

Solution

  • Looking at the library code, it looks like hypothesis should be an iterable. https://www.nltk.org/_modules/nltk/translate/meteor_score.html. The error is coming from:

    if isinstance(hypothesis, str):
            raise TypeError(
                f'"hypothesis" expects pre-tokenized hypothesis (Iterable[str]): {hypothesis}'
            )
    

    Try putting "an apple on this tree" in a list.