I'm learning to get the confidence intervals of performance metrics following this article: Confidence Intervals for Performance Metrics
but I always got this error when I run mars_boot <- int_pctl(mars_res, alpha = 0.10):
mars_boot <- int_pctl(mars_res, alpha = 0.10)
Error in UseMethod("int_pctl") :
no applicable method for 'int_pctl' applied to an object of class "c('tune_results', 'tbl_df', 'tbl', 'data.frame')"
I checked the document of int_pctl, it seems that it only supports the bootstrap resamples created by bootstraps().
This is my session info:
sessionInfo()
R version 4.3.2 (2023-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 11 x64 (build 22631)
Matrix products: default
locale:
[1] LC_COLLATE=Chinese (Simplified)_China.utf8
[2] LC_CTYPE=Chinese (Simplified)_China.utf8
[3] LC_MONETARY=Chinese (Simplified)_China.utf8
[4] LC_NUMERIC=C
[5] LC_TIME=Chinese (Simplified)_China.utf8
time zone: Asia/Shanghai
tzcode source: internal
attached base packages:
[1] stats graphics grDevices utils datasets methods
[7] base
other attached packages:
[1] earth_5.3.2 plotmo_3.6.2 TeachingDemos_2.12
[4] plotrix_3.8-4 Formula_1.2-5 yardstick_1.3.1
[7] workflowsets_1.0.1 workflows_1.1.4 tune_1.1.2
[10] tidyr_1.3.0 tibble_3.2.1 rsample_1.2.1
[13] recipes_1.0.9 purrr_1.0.2 parsnip_1.2.1
[16] modeldata_1.3.0 infer_1.0.5 ggplot2_3.5.1
[19] dplyr_1.1.4 dials_1.2.0 scales_1.3.0
[22] broom_1.0.5 tidymodels_1.1.1
loaded via a namespace (and not attached):
[1] gtable_0.3.4 lattice_0.21-9 vctrs_0.6.5
[4] tools_4.3.2 generics_0.1.3 parallel_4.3.2
[7] fansi_1.0.6 pkgconfig_2.0.3 Matrix_1.6-1.1
[10] data.table_1.15.2 lhs_1.1.6 GPfit_1.0-8
[13] lifecycle_1.0.4 compiler_4.3.2 munsell_0.5.0
[16] codetools_0.2-19 DiceDesign_1.10 class_7.3-22
[19] prodlim_2023.08.28 modelenv_0.1.1 pillar_1.9.0
[22] furrr_0.3.1 MASS_7.3-60 gower_1.0.1
[25] iterators_1.0.14 rpart_4.1.21 foreach_1.5.2
[28] parallelly_1.36.0 lava_1.7.3 tidyselect_1.2.0
[31] digest_0.6.33 future_1.33.0 listenv_0.9.0
[34] splines_4.3.2 grid_4.3.2 colorspace_2.1-0
[37] cli_3.6.2 magrittr_2.0.3 survival_3.5-7
[40] utf8_1.2.4 future.apply_1.11.0 withr_2.5.2
[43] backports_1.4.1 lubridate_1.9.3 timechange_0.2.0
[46] globals_0.16.2 nnet_7.3-19 timeDate_4032.109
[49] hardhat_1.3.0 rlang_1.1.2 Rcpp_1.0.11
[52] glue_1.6.2 ipred_0.9-14 rstudioapi_0.15.0
[55] R6_2.5.1
How should I solve this? And is there any other easy way to calculate the confidence intervals of performance metrics?
Thank you very much.
You’ll need version 1.2.0 (or later) of tune. I suggest the current version (1.2.1).
Well add a note about that to the article.