pythonscikit-learnlogistic-regressionadaboost

How to get the coefficients of the model using sklearn's AdaBoostClassifier (with Logistic regression as the base estimator)

I have built a model using scikit-learn's AdaBoostClassifier with Logistic regression as the base estimator.

model = AdaBoostClassifier(base_estimator=linear_model.LogisticRegression()).fit(X_train, Y_train)

How do I obtain the coefficients of the model? I want to see how much each feature will contribute numerically towards the target variable log(p/(1-p)).

Many thanks.

Solution

  • Adaboost have an estimators_ attribute that allows you to iterate over all fitted base learners. And, you can use coef_ parameter of each base learner to get the coefficients assigned to each feature. You can then average the coefficients. Note, you'll have to take into account the fact that Adaboost's base learners are assigned individual weight.

    coefs = []
for clf,w in zip(model.estimators_,model.estimator_weights_):
    coefs.append(clf.coef_*w)
coefs = np.array(coefs).mean(axis=0)
print(coefs)

    If you've got binary classification, you might want to change the line inside the loop as:

    coefs.append(clf.coef_.reshape(-1)*w)