Degrees in Support Vector Regression - RBF Kernel

Question

I would like to ask about the RBF Kernel on SVM.

In sklearn's documentation over here: http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVR.html#sklearn.svm.SVR it is stated that "degree of kernel function is significant only in poly, rbf, sigmoid.

I can understand the meaning of degrees on a polynomial kernel, but what about the gaussian (rbf) kernel? As I can see, the default value is 3 in sklearn's library. I also ran a GridSearch with some numbers I came up with, which estimated 3 as the best value too. Is it really significant or is this just a misstype? If so, can someone please explain the meaning and value of it?

Thanks in advance

Answer 1

A kernel is just a basis function with which you implement your model. A polynomial function of degree 3 is ax^3+bx^2+cx+d. You can use polynomials of higher degrees, however you might get overfitting, which means that your model do not generalize well, which is exactly what you want. There are several techniques to prevent overfitting.

A RBF kernel is based on gaussin functions, something like aexp(-bx). If you don't know anything about machine learning I recommend to use these ones. Generally they adapt the best.

If you want more information about machine learning, Ng's course on coursera is very good for beginners.