pythonfunctionmachine-learningconfusion-matrixmulticlass-classification

How to extract performance metrics from confusion matrix for multiclass classification model with 5 classes in Python?


I built multiclass classification model (with 5 classes in target) in Python and I have confusion matrix like below:

confusion_matrix(y_test, model.predict(X_test))

[[2006 114 80 312 257]
 [567  197 87 102 155]
 [256  84  316 39 380]
 [565  30  67 592 546]
 [363  71  186 301 1402]]

How can I calculate based on confusion matrix above, the following values:

  1. True Negative
  2. False Positive
  3. False Negative
  4. True Positive
  5. Accuracy
  6. True Positive Rate
  7. False Positive Rate
  8. True Negative Rate
  9. False Negative Rate

I have the following function to calculate that for binnary target, but how can I modify that function to calculate that for my 5 classes target ?

def xx(model, X_test, y_test):
    CM = confusion_matrix(y_test, model.predict(X_test))
    print(CM)

    print("-"*40)

    TN = CM[0][0]
    FP = CM[0][1]
    FN = CM[1][0]
    TP = CM[1][1]
    sensitivity=TP/float(TP+FN)
    specificity=TN/float(TN+FP)

    print("True Negative:", TN)
    print("False Positive:", FP)
    print("False Negative:", FN)
    print("True Positive:", TP)
    print("Accuracy", round((TN + TP) / len(model.predict(X_test)) * 100, 2), "%")
    print("True Positive rate",round(TP/(TP+FN)*100,2), "%")
    print("False Positive rate",round(FP/(FP+TN)*100,2), "%")
    print("True Negative rate",round(TN/(FP+TN)*100,2), "%")
    print("False Negative rate",round(FN/(FN+TP)*100,2), "%")

Solution

  • You have to compute N confusion matrices (N is the number of classes) to have binary matrix (One class VS. Others):

    def confusion_matrix_for(cls, cm):
        TP = cm[cls, cls]
        FN = cm[cls].sum() - TP
        FP = cm[:, cls].sum() - TP
        TN = cm.sum() - TP - FN - FP
        return np.array([[TP, FN], [FP, TN]])
    

    Usage:

    # Confusion matrix for class 0
    >>> confusion_matrix_for(0, CM)
    array([[2006,  763],    # TP | FN
           [1751, 4555]])   # FP | TN
    
    # Flatten confusion matrix for class 0
    >>> confusion_matrix_for(0, CM).ravel()
    array([2006,  763, 1751, 4555])  # TP, FN, FP, TN
    

    Use a loop and your code to compute your metrics for each class.

    Update

    CM = confusion_matrix(y_test, model.predict(X_test))
    
    for cls in range(CM.shape[0]):
        print(f'[Class {cls} vs others]')
        TP, FN, FP, TN = confusion_matrix_for(cls, CM).ravel()
        print(f'TP: {TP}, FN: {FN}, FP: {FP}, TN: {TN}')
        # compute your metrics (your code in the question)
        print()
    
    # Output
    [Class 0 vs others]
    TP: 2006, FN: 763, FP: 1751, TN: 4555
    
    [Class 1 vs others]
    TP: 197, FN: 911, FP: 299, TN: 7668
    
    [Class 2 vs others]
    TP: 316, FN: 759, FP: 420, TN: 7580
    
    [Class 3 vs others]
    TP: 592, FN: 1208, FP: 754, TN: 6521
    
    [Class 4 vs others]
    TP: 1402, FN: 921, FP: 1338, TN: 5414