python-3.xscikit-learnsvmgrid-searchgridsearchcv

Grid_search (RandomizedSearchCV) extremely slow with SVM (SVC)

I'm testing hyperparameters for an SVM, however, when I resort to Gridsearch or RandomizedSearchCV, I haven't been able to get a resolution, because the processing time is exceeding hours.

My dataset is relatively small: 4303 rows and 67 attributes, with four classes (classification problem)

Here are the tested parameters:

params =[{'C': [0.1,1, 10, 100], 
         'kernel': ['poly','sigmoid','linear','rbf'],
         'gamma': [1,0.1,0.01,0.001]}
]

sv = SVC()

clf = RandomizedSearchCV(estimator=sv,
                   cv = 3, 
                   param_distributions=params,
                   scoring='accuracy', 
                   n_jobs = -1,
                   verbose=1)
clf.fit(X, y)
print("Best parameters:", clf.best_params_)
print("better accuracy: ", (clf.best_score_)**(1/2.0))

I've already reduced the number of parameters and the number of cvs, but I still can't get a result that doesn't take hours of processing.

Is it possible to optimize this process? Am I making a mistake regarding gridsearch or SVM?

Solution

  • some SVM kernels can fit for a long time with a lot of features. I suppose you can try to remove some kernels or reduce the number of features