I want to predict result by meta-features made from stacking with original features.
I have used mlxtend for stacking, and I tried to use original features with meta-features but this library can't work well.
from lightgbm import LGBMRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.datasets import load_boston
from mlxtend.regressor import StackingRegressor
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.pipeline import make_pipeline
from sklearn.model_selection import cross_validate
boston= load_boston()
y = boston['target']
X = boston['data']
class extAll(BaseEstimator, TransformerMixin):
def __init__(self):
pass
def fit(self, X, y=None):
return self
def transform(self, X):
return self
def predict(self, X):
return self
RF = RandomForestRegressor()
LGBM = LGBMRegressor()
pipe = make_pipeline(extAll())
stack1 = StackingRegressor(regressors=[RF,LGBM,pipe], meta_regressor=LGBM, verbose=1)
scores = cross_validate(stack1, X, y, cv=10)
and errors occured as
Fitting 3 regressors...
Fitting regressor1: randomforestregressor (1/3)
Fitting regressor2: lgbmregressor (2/3)
Fitting regressor3: pipeline (3/3)
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2963, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "C:\ProgramData\Anaconda3\lib\site-packages\mlxtend\regressor\stacking_regression.py", line 154, in fit
meta_features = self.predict_meta_features(X)
File "C:\ProgramData\Anaconda3\lib\site-packages\mlxtend\regressor\stacking_regression.py", line 221, in predict_meta_features
return np.column_stack([r.predict(X) for r in self.regr_])
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\lib\shape_base.py", line 369, in column_stack
return _nx.concatenate(arrays, 1)
ValueError: all the input array dimensions except for the concatenation axis must match exactly
I think it was caused by original data which had multi-dimensional.
I want to know a better method or tools.
What should I do?
The code has some mistake in the part of the prediction. It should be correct as
class extAll(BaseEstimator, TransformerMixin,RegressorMixin):
def __init__(self):
pass
def fit(self, X, y=None):
return self
def transform(self, X):
return self
def predict(self, X):
return X
When we develop a scikit-learn type method, RegressorMixin or ClassifierMixin is needed for predictions. This code works well.