Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to do multi class classification using Support Vector Machines (SVM)

In every book and example always they show only binary classification (two classes) and new vector can belong to any one class.

Here the problem is I have 4 classes(c1, c2, c3, c4). I've training data for 4 classes.

For new vector the output should be like

C1 80% (the winner)

c2 10%

c3 6%

c4 4%

How to do this? I'm planning to use libsvm (because it most popular). I don't know much about it. If any of you guys used it previously please tell me specific commands I'm supposed to use.

like image 830
mlguy Avatar asked Dec 24 '09 13:12

mlguy


People also ask

Can I use SVM for multiclass classification?

In its most basic type, SVM doesn't support multiclass classification. For multiclass classification, the same principle is utilized after breaking down the multi-classification problem into smaller subproblems, all of which are binary classification problems.

How does SVM SVC handle multiple classes?

In its most simple type, SVM doesn't support multiclass classification natively. It supports binary classification and separating data points into two classes. For multiclass classification, the same principle is utilized after breaking down the multiclassification problem into multiple binary classification problems.

Which is the best multiclass SVM method?

The results of this paper indicate that PWC PSVM is the best single kernel discriminant method for solving multiclass problems.

Is SVM only for two classes?

Yes, support vector machines were originally designed to only support two-class-problems. That is not only true for linear SVMs, but for support vector machines in general.


1 Answers

LibSVM uses the one-against-one approach for multi-class learning problems. From the FAQ:

Q: What method does libsvm use for multi-class SVM ? Why don't you use the "1-against-the rest" method ?

It is one-against-one. We chose it after doing the following comparison: C.-W. Hsu and C.-J. Lin. A comparison of methods for multi-class support vector machines, IEEE Transactions on Neural Networks, 13(2002), 415-425.

"1-against-the rest" is a good method whose performance is comparable to "1-against-1." We do the latter simply because its training time is shorter.

like image 80
Oben Sonne Avatar answered Oct 07 '22 02:10

Oben Sonne