I am trying to build a Word2vec model but when I try to reshape the vector for tokens, I am getting this error. Any idea ?
wordvec_arrays = np.zeros((len(tokenized_tweet), 100))
for i in range(len(tokenized_tweet)):
wordvec_arrays[i,:] = word_vector(tokenized_tweet[i], 100)
wordvec_df = pd.DataFrame(wordvec_arrays)
wordvec_df.shape
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-101-71156bf1c4a3> in <module>
1 wordvec_arrays = np.zeros((len(tokenized_tweet), 100))
2 for i in range(len(tokenized_tweet)):
----> 3 wordvec_arrays[i,:] = word_vector(tokenized_tweet[i], 100)
4 wordvec_df = pd.DataFrame(wordvec_arrays)
5 wordvec_df.shape
<ipython-input-100-e3a82e60af93> in word_vector(tokens, size)
4 for word in tokens:
5 try:
----> 6 vec += model_w2v[word].reshape((1, size))
7 count += 1.
8 except KeyError: # handling the case where the token is not in vocabulary
TypeError: 'Word2Vec' object is not subscriptable
As of Gensim 4.0 & higher, the Word2Vec
model doesn't support subscripted-indexed access (the ['...']') to individual words. (Previous versions would display a deprecation warning,
Method will be removed in 4.0.0, use self.wv.getitem() instead`, for such uses.)
So, when you want to access a specific word, do it via the Word2Vec
model's .wv
property, which holds just the word-vectors, instead. So, your (unshown) word_vector()
function should have its line highlighted in the error stack changed to:
vec += model_w2v.wv[word].reshape((1, size))