rproc-r-package

Getting numbers of true positives, negatives, likelihood ratio from a pROC ROC curve

I have an analysis using pROC package to assess the performance of a blood test for detection of an outcome (binary). I have used pROC and bootstrapping to select the optimum threshold (using closest top left) and to give sensitivity and specificity at thresholds to determine 90 and 95% sensitivity.

I would like to determine the likelihood ratio (and 95% CI) at each threshold, anyone have any tips for getting this out of pROC with 95% CI (bootstrapped)

Reproducible example:

set.seed(42)

num_subjects <- 40

continuous_variable <- rnorm(num_subjects, mean = 50, sd = 10)

probability_of_one <- 1 / (1 + exp(-0.1 * (continuous_variable - 50)))
binary_variable <- rbinom(num_subjects, 1, probability_of_one)

df <- data.frame(
  continuous_variable = continuous_variable,
  binary_variable = binary_variable
)

print(df)

Current attempt:

ROC_1 <- pROC::roc(response = df$binary_variable, predictor = df$continuous_variable, ci=TRUE)
pROC::auc(ROC_1) #0.9247
pROC::ci.auc(ROC_1, conf.level=0.95, method=c("bootstrap"), boot.n = 2000) 
d <- coords(ROC_1, x="all")
threshold <- coords(ROC_1, x="best", best.method=c("closest.topleft"))$threshold

pROC::coords(ROC_1 , x=threshold, ret=c("tn", "tp", "fn", "fp"))
pROC::ci.coords(ROC_1 , x=-threshold, ret=c("tn", "tp", "fn", "fp"))

Note THRESHOLD is manually inputted depending on the threshold desired

Solution

  • The likelihood ratios are defined as:

    LR+ = sensitivity / (1 - specificity)

    LR+ = (1 - sensitivity) / specificity

    You can compute them easily from the output of the coords function:

    d <- coords(ROC_1, x="all")
d$lr_pos = d$sensitivity / (1 - d$specificity)
d$lr_neg = (1 - d$sensitivity) / 1 - d$specificity

    Computing 95% CI is not possible at this time, but I will consider it for a future version of pROC.