rr-carethyperparameters

Adaptive Resampling hyperparameter tuning algorithm (caret package) for 'mlpWeightDecayML' and 'glmnet'

I am comparing different hyperparameter tuning algorithms in R and I have been trying the Adaptive Random Sampling from the caret package. I am encountering a weird bug when trying the caret models 'mlpWeightDecayML' and 'glmnet'. The smallest code snippet that produces the exception:

control = caret::trainControl(
  method = "adaptive_cv",
  number = 10, repeats = 3,
  adaptive = list(min = 5, alpha = 0.05, 
                  method = "gls", complete = TRUE),
  search = "random",
)  

caretModel <- caret::train(
  x = iris[, -5], 
  y = iris[, 5],
  method = "mlpWeightDecayML",
  trControl = control,
  tuneLength = 10
)

Error that keeps happening every once in a while (without changing anything) is:

Error in { : task 6 failed - "arguments imply differing number of rows: 0, 15"

I have tried different datasets and it gives the same error, just different rows in the error message. Also, the error doesn't happen with the models 'ranger' and 'xgbTree' for example.

I have tried changing R versions (4.3.2, 4.3.1, 4.2.2) but the same error occurs.

Solution

  • If we set.seed before running the model, your code works fine, like

    library(caret)

control = caret::trainControl(
  method = "adaptive_cv",
  number = 10, repeats = 3,
  adaptive = list(min = 5, alpha = 0.05, 
                  method = "gls", complete = TRUE),
  search = "random"
) 

set.seed(825)
caretModel <- caret::train(
  x = iris[, -5], 
  y = iris[, 5],
  method = "mlpWeightDecayML",
  trControl = control,
  preProc = c("center", "scale"),
  metric = "Accuracy",
  tuneLength = 10
)