machine-learningclassificationpredictionxgboosteli5

Problem with XGboost Classification & eli5 package


When training an XGBoost classification model, I am using the eli5 function "explain_prediction()" to look at the feature contributions to invidividual predictions.

However, the eli5 package seems to be treating my model as a regressor rather than a classifier.

Below is a snippet of code, showing my model, my prediction, and then the output from the "explain_prediction" method. enter image description here

As you can see, the output gives a score that is 3.016 rather than a probability between 0 and 1. In this case I would have expected 0.953.

Any help appreciated.


Solution

  • the eli5 package seems to be treating my model as a regressor rather than a classifier.

    The boosting score is converted to the probability score by applying the inverse logit function to it.

    The probability scale is non-linear, which would make the numeric interpretation of feature contributions more difficult.

    .. the output gives a score is 3.016 .. I would have expected 0.953

    1 / (1 + exp(-3.016)) = 0.9532917416863492