pythongensimword2vec

topic coherence (w2v) and its trend?


I tried to use w2v topic coherence score to evaluate the topic model based on NMF. Below is the w2v coherences I have calculated.

And I want to know, is w2v coherence higher better? Also, why the coherence scores get lower with more topics?

w2v scores with different topic numbers:

w2v scores with different topic numbers


Solution

  • Is w2v coherence better if it's higher?

    Yes, generally a higher w2v coherence score means better topic quality. It basically says, that the words that make up each topic are more coherent.

    Why does the coherence scores get lower with more topics?

    Usually as you increase the number of topics, the model tries to divide the words into more and more groups. That then leads to less meaningful or more fragmented topics overall. That's a common trade off in topic modeling, which is why you might wanna look into other metrics like perplexity in addition to your w2v (or others, depending on your dataset)