[SOLVED] setting a level in the ROC function

setting a level in the ROC function

I am trying to obtain sens, spec, npv and ppv for the ability of two patient scoring systems and their ability to predict admission. To explain the data:

'Etriage' is electronic triage system (scores from 1-5), AETriage is a face-to-face triage system (socres 1-5). Admitted has 2 levels as factors 0 - not admitted, 1 - admitted.

eg:

df <- data.frame(
    Etriage=c(2,3,3,5,2,5,3,3,4,3),
    AETriage=c(3,4,4,3,2,4,4,3,4,1), 
    Admitted=c(1,0,0,0,0,0,0,0,0,1))

I want to compare the two scores ability to predict admission and have been trying to use pROC to do this (taken from the overall dataset):

library(pROC)
auc(df$Admitted, df$AETriage)

ci.auc(df$Admitted, df$AETriage)
ROC_AETriage_Admitted<-roc(df$Admitted,df$AETriage)
print(ROC_AETriage_Admitted)

OperatingValuesAETriage <- coords(ROC_AETriage_Admitted, "all", 
    ret=c("sensitivity","specificity","npv","ppv","threshold"), transpose = FALSE)

print(OperatingValuesAETriage)

How would I code 2 operating levels (LOW acuity = score 3-5 and HIGH acuity = 1-2) and assess their ability to predict admission?

Thanks!

Solution

You cannot use a textual score (or factors), because in order to build a ROC curve we need to rank the data, which cannot be done automatically on simple text (is "High" higher than "Low"? A computer wouldn't know that).

However, R has a textual data type for that: ordered factors, which are supported (to some extent) by pROC:

df$Etriage_binary <- ordered(
    ifelse(df$Etriage <= 2, "High", "Low"),
    levels = c("Low", "High"))

pROC::auc(df$Admitted, df$Etriage_binary)

Although thresholds aren't reported accurately, it works, but this dichotomization is lossy and there are many reasons why you don't actually want to do it.