pythonnlpnltksentiment-analysisvader

Is it possible to edit NLTK's vader sentiment lexicon?


I would like to add words to the vader_lexicon.txt to specify polarity scores to a word. What is the right way to do so?

I saw this file in AppData\Roaming\nltk_data\sentiment\vader_lexicon. The file consists of the word, its polarity, intensity, and an array of 10 intensity scores given by "10 independent human raters". [1] However, when I edited it, nothing changed in the results of the following code:

from nltk.sentiment.vader import SentimentIntensityAnalyzer
sia = SentimentIntensityAnalyzer()
s = sia.polarity_scores("my string here")

I think that this text file is accessed by my code when I called SentimentIntensityAnalyzer's constructor. [2] Do you have any ideas on how I can edit a pre-made lexicon?

Sources:

[1] https://github.com/cjhutto/vaderSentiment

[2] http://www.nltk.org/api/nltk.sentiment.html


Solution

  • For anyone interested, this can also be achieved without having to manually edit the vader lexicon .txt file. Once loaded the lexicon is a normal dictionary with words as keys and scores as values. As provided by repoleved in this post:

    from nltk.sentiment.vader import SentimentIntensityAnalyzer
    
    new_words = {
        'foo': 2.0,
        'bar': -3.4,
    }
    
    SIA = SentimentIntensityAnalyzer()
    
    SIA.lexicon.update(new_words)
    

    If you wish to remove words, use the '.pop' function:

    SIA = SentimentIntensityAnalyzer()
    
    SIA.lexicon.pop('no')