How to fix IndexError: invalid index to scalar variable

This code generates error:

IndexError: invalid index to scalar variable.

at the line: results.append(RMSPE(np.expm1(y_train[testcv]), [y[1] for y in y_test]))

How to fix it?

import pandas as pd
import numpy as np
from sklearn import ensemble
from sklearn import cross_validation

def ToWeight(y):
    w = np.zeros(y.shape, dtype=float)
    ind = y != 0
    w[ind] = 1./(y[ind]**2)
    return w

def RMSPE(y, yhat):
    w = ToWeight(y)
    rmspe = np.sqrt(np.mean( w * (y - yhat)**2 ))
    return rmspe

forest = ensemble.RandomForestRegressor(n_estimators=10, min_samples_split=2, n_jobs=-1)

print ("Cross validations")
cv = cross_validation.KFold(len(train), n_folds=5)

results = []
for traincv, testcv in cv:
    y_test = np.expm1(forest.fit(X_train[traincv], y_train[traincv]).predict(X_train[testcv]))
    results.append(RMSPE(np.expm1(y_train[testcv]), [y[1] for y in y_test]))

testcv is:

[False False False ...,  True  True  True]

Solution

You are trying to index into a scalar (non-iterable) value:

[y[1] for y in y_test]
#  ^ this is the problem

When you call [y for y in test] you are iterating over the values already, so you get a single value in y.

Your code is the same as trying to do the following:

y_test = [1, 2, 3]
y = y_test[0] # y = 1
print(y[0]) # this line will fail

I'm not sure what you're trying to get into your results array, but you need to get rid of [y[1] for y in y_test].

If you want to append each y in y_test to results, you'll need to expand your list comprehension out further to something like this:

[results.append(..., y) for y in y_test]

Or just use a for loop:

for y in y_test:
    results.append(..., y)