pythonxgboosthyperoptxgbregressor

Error while hyper parameter tuning of XGBRegressor in xgboost


I have been trying tune my XGBoost model in order to predict values of a target column, using the xgboost and hyperopt library in python. After importing the required libraries correctly, the domain space, objective function and running the optimization step as follows:

space= { 'booster': 'gbtree',#hp.choice('booster',['gbtree','dart']),    
         'max_depth': hp.choice('max_depth',[i for i in range(3,18,1)]),   
         'gamma':0.2,
         'colsample_bytree':hp.choice('colsample_bytree',[ 0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0 ]),
         'min_child_weight' :hp.choice('min_child_weight',[ 1, 3, 5, 7 ]),
         'learning_rate':hp.choice('learning_rate',[0.05, 0.10, 0.15, 0.20, 0.25, 0.30 ]),
         'n_estimators': 500,
         'seed': 0,
         'objective':'reg:linear',
       }

def objective(space):
    reg = xgb.XGBRegressor(space)
    reg.fit(X_train, y_train,
        eval_set=[(X_train, y_train), (X_test, y_test)],
        verbose=100)
    preds = reg.predict(X_test)
    score = np.sqrt(mean_squared_error(test['Close'], test['prediction']))
    print(f'RMSE Score on Test set: {score:0.2f}')
    return {'RMSE': score, 'status': STATUS_OK }

trials=Trials()
best_hyper=fmin(fn = objective,
                space = space,
                algo = tpe.suggest,
                max_evals = 100,
                trials = trials)

On execution, I get the following error:

XGBoostError: [14:08:49] C:\Users\Administrator\workspace\xgboost-win64_release_1.6.0\src\objective\objective.cc:26: Unknown objective function: `{'booster': 'gbtree', 'colsample_bytree': 1.0, 'gamma': 0.2, 'learning_rate': 0.05, 'max_depth': 16, 'min_child_weight': 7, 'n_estimators': 500, 'objective': 'reg:linear', 'seed': 0}`
Objective candidate: survival:aft
Objective candidate: binary:hinge
Objective candidate: multi:softmax
Objective candidate: multi:softprob
Objective candidate: rank:pairwise
Objective candidate: rank:ndcg
Objective candidate: rank:map
Objective candidate: survival:cox
Objective candidate: reg:gamma
Objective candidate: reg:squarederror
Objective candidate: reg:squaredlogerror
Objective candidate: reg:logistic
Objective candidate: binary:logistic
Objective candidate: binary:logitraw
Objective candidate: reg:tweedie
Objective candidate: reg:linear
Objective candidate: reg:pseudohubererror
Objective candidate: count:poisson

How do I debug and resolve this error?I referred to the documentation but couldn't understand the issue.


Solution

  • I found a solution to this. I simply had to pass the parameters preceded with ** like this in the objective function:

    reg = xgb.XGBRegressor(**space)
    

    Hope this is helpful!