Both GPRegression
(GPy
) and GaussianProcessRegressor
(scikit-learn
) uses similar initial values and the same optimizer (lbfgs
). Why results vary significantly?
#!pip -qq install pods
#!pip -qq install GPy
from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF, ConstantKernel as C
from sklearn.preprocessing import StandardScaler
import pods
data = pods.datasets.olympic_marathon_men()
X = StandardScaler().fit_transform(data['X'])
y = data['Y']
# scikit-learn
model = GaussianProcessRegressor(C()*RBF(), n_restarts_optimizer=20, random_state=0)
model.fit(X, y)
print(model.kernel_)
# GPy
from GPy.models import GPRegression
from GPy.kern import RBF as GPyRBF
model = GPRegression(X, y, GPyRBF(1))
model.optimize_restarts(20, verbose=0)
print(model.kern)
Results
2.89**2 * RBF(length_scale=0.173)
rbf. | value | constraints | priors
variance | 25.399509298957504 | +ve |
lengthscale | 4.279767394389103 | +ve |
Using GPy RBF()
kernel is equivalent to using scikit-learn ConstantKernel()*RBF() + WhiteKernel()
. Because GPy library adds likelihood noise internally. Using this I was able to get comparable results in both.