roptimizationhyperparametersmlrhyperopt

How to use mlrMBO with mlr for hyperparameter optimisation and tuning

Im trying to train ML algorithms (rf, adaboost, xgboost) in R on a dataset where the target is multiclass classification. For hyperparameter tuning I use the MLR package.

My goal of the code below is to tune the parameters mtry and nodesize, but keep ntrees constant at 128 (with mlrMBO). However, I get the error message below. How can I define this in the correct way?

rdesc <- makeResampleDesc("CV",stratify = T,iters=10L) 

traintask <- makeClassifTask(data = df_train, 
                             target = "more_than_X_perc_damage")
testtask <- makeClassifTask(data = df_test, 
                            target = "more_than_X_perc_damage")

lrn <- makeLearner("classif.randomForest",
                   predict.type = "prob")

# parameter space 
params_to_tune <- makeParamSet(makeIntegerParam("ntree", lower = 128, upper = 128),
                               makeNumericParam("mtry", lower = 0, upper = 1, trafo = function(x) ceiling(x*ncol(train_x))),
                               makeNumericParam("nodesize",lower = 0,upper = 1, trafo = function(x) ceiling(nrow(train_x)^x)))

ctrl = makeTuneControlMBO(mbo.control=mlrMBO::makeMBOControl())
tuned_params <- tuneParams(learner = lrn, 
                           task = traintask, 
                           control = ctrl, 
                           par.set = params_to_tune, 
                           resampling = rdesc, 
                           measure=acc)

rf_tuned_learner <- setHyperPars(learner = lrn,
                                 par.vals = tuned_params$x)

rf_tuned_model <- mlr::train(rf_tuned_learner, traintask)

# prediction performance
pred <- predict(rf_tuned_model, testtask)
performance(pred) 
calculateConfusionMatrix(pred)
stats <- confusionMatrix(pred$data$response,pred$data$truth)
acc_rf_tune <- stats$overall[1] # accuracy
print(acc_rf_tune)

Error in (function (fn, nvars, max = FALSE, pop.size = 1000, max.generations = 100, : Domains[,1] must be less than or equal to Domains[,2]

Thanks in advance!

Solution

  • You can do this by not including the hyperparameter you want to keep constant in the ParamSet and instead setting it to the value you want when creating the learner.