This is my SVR to learn adder function (y=x1 + x2):
%reset -f
#Libs
from sklearn import svm;
#PROGRAMME ENTRY POINT==========================================================
#Data, addition
#Exp[I] = sum(Inp[I])
Inp = [[1,2],[3,4],[5,6],[7,8],[9,0]];
Exp = [ 3, 7, 11, 15, 9 ];
#Train
Model = svm.SVR(kernel="poly",degree=3);
Model.fit(Inp,Exp);
#Infer
print("Input values are those in the train data:");
print(f"1 + 2 = {Model.predict([[1,2]])[0]:.6f}");
print("\nInput values are those in the train data:");
print(f"5 + 6 = {Model.predict([[5,6]])[0]:.6f}");
print("\nInput values are those NOT in the train data, but in range:");
print(f"5 + 5 = {Model.predict([[5,5]])[0]:.6f}");
print("\nInput values are those NOT in the train data, and OUT of range:");
print(f"9 + 1 = {Model.predict([[9,1]])[0]:.6f}");
#EOF
But the result isn't what expected for:
Input values are those in the train data:
1 + 2 = 6.007171
Input values are those in the train data:
5 + 6 = 9.595818
Input values are those NOT in the train data, but in range:
5 + 5 = 8.533934
Input values are those NOT in the train data, and OUT of range:
9 + 1 = 9.170507
Is it possible for sklearn SVR to generalise the adder function? What should be changed in the code above to make SVR learn x1+x2?
Polynomial kernel of third degree simply have too much variance to correctly predict such a simple function, especially on such a small dataset. This is based on bias/variance tradeoff. Your model is loosing on variance while getting almost nothing on bias in this case (your function is overly complex). The same goes even for lower order polynomials and radial basis function.
Lowering the variance of the model will do the trick. Just use linear kernel.
Model = svm.SVR(kernel="linear")
Results for SVM with linear kernel are:
Input values are those in the train data:
1 + 2 = 3.100000
Input values are those in the train data:
5 + 6 = 10.966667
Input values are those NOT in the train data, but in range:
5 + 5 = 9.983333
Input values are those NOT in the train data, and OUT of range:
9 + 1 = 9.983333