Trying to use all cores for this apply function using modin.pandas
from nltk.sentiment.vader import SentimentIntensityAnalyzer
sid = SentimentIntensityAnalyzer()
# sentiment Score of essay
data = data.merge(data.essay.apply(lambda s: pd.Series({'neg':sid.polarity_scores(s)['neg'],
'neu':sid.polarity_scores(s)['neu'],
'pos':sid.polarity_scores(s)['pos'],
'compound':sid.polarity_scores(s)['compound']})),
left_index=True, right_index=True)
It works with default pandas, but using modin raises this error:
ValueError: can not merge DataFrame with instance of type <class 'modin.pandas.series.Series'>
essay is text column in the DataFrame named "data"
As the answers to this question suggest, you are likely getting this error because you are merging a pandas.Dataframe with a modin.Series. For your example, try converting data to a modin dataframe with modin.pandas.DataFrame(data).