I am trying to tune parameters for my model using GridSearchCV. However, I keep getting the same error telling me that the parameter grid that I am passing contains parameters that are invalid. for example, it keeps telling me invalid parameter learning_rate for estimator...... This is happening not just for Adaboost but all other models that I have tried to tune using GridsearchCV, that is, Logistic Regression, LinearSVC, Decision Trees and RandomForest. Here is the code that I wrote for Adaboost classifier and the errors that I am getting:
clf_adaboost = Pipeline([('vect', CountVectorizer()),('tfidf', TfidfTransformer()),('clf', AdaBoostClassifier())
])
clf = Pipeline([
('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', clf_adaboost)
])
parameters = {
'n_estimators': [20, 50, 70, 100],
'learning_rate' : [0.0001, 0.001, 0.01, 0.1, 0.2, 0.3],
'n_estimators' : [100, 200, 300, 400, 500]
}
kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=7)
gs_clf = GridSearchCV(clf, parameters, cv=kfold, n_jobs=-1)
gs_clf = gs_clf.fit(twenty_train.data, twenty_train.target)
print("Best score accurracy = %.3f%%" %((gs_clf.best_score_)*100.0))
print("Best parameters are : ")
print(gs_clf.best_params_)
when i run this, I get the following error:
oblib.externals.loky.process_executor._RemoteTraceback:
"""
Traceback (most recent call last):
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 418, in _process_worker
r = call_item()
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/externals/loky/process_executor.py", line 272, in __call__
return self.fn(*self.args, **self.kwargs)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 567, in __call__
return self.func(*args, **kwargs)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/parallel.py", line 225, in __call__
for func, args, kwargs in self.items]
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/parallel.py", line 225, in <listcomp>
for func, args, kwargs in self.items]
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/sklearn/model_selection/_validation.py", line 503, in _fit_and_score
estimator.set_params(**parameters)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/sklearn/pipeline.py", line 164, in set_params
self._set_params('steps', **kwargs)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/sklearn/utils/metaestimators.py", line 50, in _set_params
super().set_params(**params)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/sklearn/base.py", line 224, in set_params
(key, self))
ValueError: Invalid parameter learning_rate for estimator Pipeline(memory=None,
steps=[('vect',
CountVectorizer(analyzer='word', binary=False,
decode_error='strict',
dtype=<class 'numpy.int64'>, encoding='utf-8',
input='content', lowercase=True, max_df=1.0,
max_features=None, min_df=1,
ngram_range=(1, 1), preprocessor=None,
stop_words=None, strip_accents=None,
token_pattern='(?u)\\b\\w\\w+\\b',
tokenizer=None, vocabulary=Non...
preprocessor=None,
stop_words=None,
strip_accents=None,
token_pattern='(?u)\\b\\w\\w+\\b',
tokenizer=None,
vocabulary=None)),
('tfidf',
TfidfTransformer(norm='l2', smooth_idf=True,
sublinear_tf=False,
use_idf=True)),
('clf',
AdaBoostClassifier(algorithm='SAMME.R',
base_estimator=None,
learning_rate=1.0,
n_estimators=50,
random_state=None))],
verbose=False))],
verbose=False). Check the list of available parameters with `estimator.get_params().keys()`.
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "twenty_news.py", line 61, in <module>
gs_clf = gs_clf.fit(twenty_train.data, twenty_train.target)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/sklearn/model_selection/_search.py", line 688, in fit
self._run_search(evaluate_candidates)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/sklearn/model_selection/_search.py", line 1149, in _run_search
evaluate_candidates(ParameterGrid(self.param_grid))
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/sklearn/model_selection/_search.py", line 667, in evaluate_candidates
cv.split(X, y, groups)))
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/parallel.py", line 934, in __call__
self.retrieve()
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/parallel.py", line 833, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/site-packages/joblib/_parallel_backends.py", line 521, in wrap_future_result
return future.result(timeout=timeout)
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/Users/Furaha/.pyenv/versions/3.7.3/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
ValueError: Invalid parameter learning_rate for estimator Pipeline(memory=None,
steps=[('vect',
CountVectorizer(analyzer='word', binary=False,
decode_error='strict',
dtype=<class 'numpy.int64'>, encoding='utf-8',
input='content', lowercase=True, max_df=1.0,
max_features=None, min_df=1,
ngram_range=(1, 1), preprocessor=None,
stop_words=None, strip_accents=None,
token_pattern='(?u)\\b\\w\\w+\\b',
tokenizer=None, vocabulary=Non...
preprocessor=None,
stop_words=None,
strip_accents=None,
token_pattern='(?u)\\b\\w\\w+\\b',
tokenizer=None,
vocabulary=None)),
('tfidf',
TfidfTransformer(norm='l2', smooth_idf=True,
sublinear_tf=False,
use_idf=True)),
('clf',
AdaBoostClassifier(algorithm='SAMME.R',
base_estimator=None,
learning_rate=1.0,
n_estimators=50,
random_state=None))],
verbose=False))],
verbose=False). Check the list of available parameters with `estimator.get_params().keys()`.
I am getting a similar error for all the models that I have tried and my guess is that I am not writing the parameters in the parameter grid correctly. Can someone help me solve this?
You are doubling the pipeline steps.
You could try it like this:
clf = Pipeline([
('vect', CountVectorizer()),
('tfidf', TfidfTransformer()),
('clf', AdaBoostClassifier())
])
parameters = {
'clf__n_estimators': [20, 50, 70, 100],
'clf__learning_rate' : [0.0001, 0.001, 0.01, 0.1, 0.2, 0.3]
}
kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=7)
gs_clf = GridSearchCV(clf, parameters, cv=kfold, n_jobs=-1)
gs_clf = gs_clf.fit(twenty_train.data, twenty_train.target)
print("Best score accurracy = %.3f%%" %((gs_clf.best_score_)*100.0))
print("Best parameters are : ")
print(gs_clf.best_params_)
refer to composite estimators and parameter spaces
In that case the Gridsearch would go through the grid for the adaboost method.
And you mentioned n_estimators twice. Were you about to optimize one of the other components in the pipe as well (but to be honest I do not think so, because none of the other has this parameter, only the adaboost-ML-method has this hyperparameter)?