I am using mlr3
and I wanted to ask if it is possible to change the resampling method of an exiting auto_tuner()
.
Example:
library(mlr3verse)
# Some existing auto_tuner
learner = lrn("classif.svm",
cost = to_tune(1e-1, 1e5),
gamma = to_tune(1e-1, 1),
kernel = "radial",
type = "C-classification"
)
at = auto_tuner(
tuner = tnr("grid_search", resolution = 5, batch_size = 5),
learner = learner,
resampling = rsmp("cv", folds = 3), # The resampling I would like to change
measure = msr("classif.ce")
)
# New resampling I would like to assign to the existing auto_tuner
new_resampling = rsmp("cv", folds = 10)
Background:
I select a model based on a nested cross validation and afterwards want to train the best model for prediction. As I use a simpler resampling inside my nested cross validatio I would like to change the resampling used by the auto_tuner
to avoid creating a new one.
I select a model based on a nested cross validation
Maybe you should think again about your approach:
Nested resampling is a statistical procedure to estimate the predictive performance of the model trained on the full dataset, it is not a procedure to select optimal hyperparameters. Nested resampling produces many hyperparameter configurations which should not be used to construct a final model
Regarding your question: No, it is not possible to change the resampling later. Just construct a new one if you want to change the resampling.