I would like to evaluate whether the logistic regression model I created is overfit. I'd like to compare the accuracies of each training fold to the test fold, but I don't know how to view these in R. This is the k-fold cross validation code:
library(caret)
levels(habitatdata$outcome) <- c("absent", "present") #rename factor levels
set.seed(12)
cvIndex <- createFolds(factor(habitatdata$outcome), 5, returnTrain = T) #create stratified folds
ctrlspecs <- trainControl(index = cvIndex,
method = "cv",
number = 5,
savePredictions = "all",
classProbs = TRUE) #specify training methods
set.seed(123)
model1 <- train(outcome~ ist + hwt,
data=habitatdata,
method = "glm",
family = binomial, trControl = ctrlspecs) #specify model
How do I view the training accuracies of each fold?
Look at model1$resample
- it should give you a table with Accuracy (and Kappa) for each fold.