Classification through Radial Basis Function (RBF) SVM

Question

I am using sklearn.svm.SVC (kernel='rbf') for the classification of an image data, which is doing pretty well job. Linear SVM classifies the data by putting a hyper plane between the two classes. In the case of rbf SVM the plane would be in infinite dimension. For any testing point we can use predict to check which it belongs to. In linear case we can manually get the prediction by getting the equation of the hyper plane. How can we do this in rbf SVM case. How exactly predict works in rbf SVM case.

Answer 1

Fisrt things first

Whenever we classify we should consider:

• Classifiers can be learnt for high dimensional features spaces, without actually having to map the points into the high dimensional space.
• Data may be linearly separable in the high dimensional space, but not linearly separable in the original feature space
• Kernels can be used for an SVM because of the scalar product in the dual form, but can also be used elsewhere – they are not tied to the SVM formalism.
• Kernels apply also to objects that are not vectors

For instance I will put some used Kernels.

For a SVM Classifier with Gaussian Kernel we would have something like:

As you notice support vector is substituted and therefore we could vary it depending on results, for example, consider two features and their colored points:

And setting some values we get:

Now

Or

Now what happens when infinity comes to play:

Then:

And what about adaBoost to play with datasets http://cseweb.ucsd.edu/~yfreund/adaboost/

If you like you could test The NETLAB ML Matlab software by Ian Nabney here

Here are more sources for SVM

• Christopher M. Bishop, "Pattern Recognition and Machine Learning" , Springer (2006), ISBN 0-38-731073-8.
• Hastie, Tibshirani, Friedman, "Elements of Statistical Learning", Second Edition, Springer, 2009. Pdf available online.
• Ian H. Witten and Eibe Frank, "Data Mining: Practical Machine Learning Tools and Techniques" , Second Edition, 2005.
• David MacKay, "Information Theory, Inference, and Learning Algorithms" Which is freely available online!
• Tom Mitchell, "Machine Learning" , McGraw Hill, 1997