I want to use cross-validation for calculating specificity. I found code for calculating accuracy, really, f1-score, and precision. but I couldn't found for specificity. for example, the code for f1-score is like:
cross_val_score(SVC, X, y, scoring="f1", cv = 7)
or for precision is like:
cross_val_score(SVC, X, y, scoring="precision", cv = 7)
The specifity is basically the True Negative Rate which is the same as the True Positive Rate (Recall) but for the negative class
If you have a binary class, you should do the following
Import the metric recall_score from metrics (details here), and make_scorer function
from sklearn.metrics import recall_score
from sklearn.metrics import make_scorer
Then you generate your new scorer, defining for which class you are calculating recall (by default, the recall is calculated on the label=1)
specificity = make_scorer(recall_score, pos_label=0)
The label 0 is usually the negative class in a binary problem.
print(cross_val_score(classifier, X_train, y_train, cv=10, specificity))
if you want the recall (True positive rate) you can do the same changing the class
sensitivity = make_scorer(recall_score, pos_label=1)
print(cross_val_score(classifier, X_train, y_train, cv=10, sensitivity))
Anyway you can make your custom scorer, if you need something more complex