scikit-learnmlxtend

Stacking with original data by mlxtend or other tools


I want to predict result by meta-features made from stacking with original features.
I have used mlxtend for stacking, and I tried to use original features with meta-features but this library can't work well.

from lightgbm import LGBMRegressor
from sklearn.ensemble import RandomForestRegressor

from sklearn.datasets import load_boston
from mlxtend.regressor import StackingRegressor
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_validate

boston= load_boston()
y = boston['target']
X = boston['data']

class extAll(BaseEstimator, TransformerMixin):
  def __init__(self):
      pass
  def fit(self, X, y=None):
      return self
  def transform(self, X):
      return self
  def predict(self, X):
      return self
RF =  RandomForestRegressor()
LGBM = LGBMRegressor()
pipe = make_pipeline(extAll())
stack1 = StackingRegressor(regressors=[RF,LGBM,pipe], meta_regressor=LGBM, verbose=1)
scores = cross_validate(stack1, X, y, cv=10)

and errors occured as

Fitting 3 regressors...
Fitting regressor1: randomforestregressor (1/3)
Fitting regressor2: lgbmregressor (2/3)
Fitting regressor3: pipeline (3/3)
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2963, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "C:\ProgramData\Anaconda3\lib\site-packages\mlxtend\regressor\stacking_regression.py", line 154, in fit
    meta_features = self.predict_meta_features(X)
  File "C:\ProgramData\Anaconda3\lib\site-packages\mlxtend\regressor\stacking_regression.py", line 221, in predict_meta_features
    return np.column_stack([r.predict(X) for r in self.regr_])
  File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\lib\shape_base.py", line 369, in column_stack
    return _nx.concatenate(arrays, 1)
ValueError: all the input array dimensions except for the concatenation axis must match exactly

I think it was caused by original data which had multi-dimensional.
I want to know a better method or tools.
What should I do?


Solution

  • The code has some mistake in the part of the prediction. It should be correct as

    class extAll(BaseEstimator, TransformerMixin,RegressorMixin):
     def __init__(self):
        pass
     def fit(self, X, y=None):
        return self
     def transform(self, X):
        return self
     def predict(self, X):
        return X
    

    When we develop a scikit-learn type method, RegressorMixin or ClassifierMixin is needed for predictions. This code works well.