watson-openscale

Question about interpretation of importance in contrastive explanation section of the explanation task response

I'm wondering how to interpret the "importance" values in the contrastive explanation part when getting an explanation task response. See https://cloud.ibm.com/apidocs/ai-openscale#get-explanation-task > entity > contrastive_explanation > pertinent_positive_features (and also pertinent_negative_features).

Importance seems to be high for some features that are not changed compared to input features, and low for some where the value has changed compared to the input. I'm trying to understand which of all the features returned are key - i first went with those that were changed compared to the input features, but since this does not line up with the importance values I'd like to understand better what this importance stands for?

For example:

"input_features": [
            {
                "feature_type": "categorical",
                "name": "CheckingStatus",
                "value": "less_0"
            },
            ...

and

 "contrastive_explanation": {
            "pertinent_positive_features": [
                ...
                {
                    "feature_value": "no_checking",
                    "importance": "0.0",
                    "feature_name": "CheckingStatus"
                },

The feature was changed and yet its importance for pertinent positive features is 0?

Solution

  • The importance of features for pertinent positive is computed as the change needed to get to the Pertinent positive feature value from its median value. E.g., if the feature F1 has value 20 in its pertinent positive and the median value of f1 is 2, then the importance depends on the distance between 20 and 2 (i.e., 18). We also factor in the standard deviation to normalise the distance across different features. Hence it could happen that a feature whose value has not changed in the PP has high weight (if that value is away from its median). So the weight is a measure of how far the PP feature value is from its median.

    Having said that, another way of looking at feature importance is to look at those features which have changed.

    Thanks, Manish