rmachine-learningr-caretprecision-recall

Is there an R function to optimize the PRG AUC (area under the precision-recall-gain curve)?

To optimize the PR AUC (area under the precision-recall curve) in Caret, the prSummary function can be plugged into the trainControl function.

Is there a similar way to optimize the PRG AUC (area under the precision-recall-gain curve), as introduced by Flach and Kull? Inside or outside of Caret?

The MLeval package returns PRG curves and PRG AUCs, but only seems to run on the Caret train output.

Solution

  • It turns out that Meelis Kull wrote an R package for the PRG AUC. Computing the PRG AUC is as simple as:

    prg_curve <- create_prg_curve(labels, scores)
prg_auc <- calc_auprg(prg_curve)

    For Caret, those functions can be plugged into a summary function, as explained by @missuse in the links in the comment above.