How to build multiclass SVM in R?

Question

I am working on the project handwritten pattern recognition(Alphabets) using Support Vector Machines. I have 26 classes in total but I am not able to classify using SVM in R. I can classify the images only if it is a binary class. How to use SVM for Multiclass SVM in R?

I am using "e1071" package.

Thanks in advance.

Answer 1

For a multi class classifier, you can get probabilities for each class. You can set 'probability = TRUE' while training the model & in 'predict' api. This will give you the probabilities of each class. Below is the sample code for iris data set:

data(iris)
attach(iris)
x <- subset(iris, select = -Species) 
y <- Species
model <- svm(x, y, probability = TRUE)
pred_prob <- predict(model, x, decision.values = TRUE, probability = TRUE)

With above code, 'pred_prob' will have probabilities among other data. You can access only probabilities in the object with below statement:

attr(pred_prob, "probabilities")

         setosa  versicolor   virginica
1   0.979989881 0.011347796 0.008662323
2   0.972567961 0.018145783 0.009286256
3   0.978668604 0.011973933 0.009357463
...

Hope this helps.

NOTE: I believe when you give 'probability' internally svm performs one vs rest classifier because, it takes lot more time with 'probability' param set against the model with 'probability' param not being set.

Answer 2

I guess the approved answer is outdated. The libsvm which is used in package e1071 supports also multi-class classification in a "1-vs.-1" model. I.e., it creates (L-choose-2) number of separation planes.

Here's an example code:

# Create 2d data
set.seed(1)
x1 = matrix(c(rnorm(20, 0), rnorm(20, 0)), ncol=2)
x2 = matrix(c(rnorm(20, 0), rnorm(20, 4)), ncol=2)
x3 = matrix(c(rnorm(20, 4), rnorm(20, 0)), ncol=2)
x4 = matrix(c(rnorm(20, 4), rnorm(20, 4)), ncol=2)
x = rbind(x1,x2,x3,x4)
y = factor(c(rep(1,20), rep(2,20), rep(3,20), rep(4,20)))

# Multiclass Classification (1 vs. 1)
fit = svm(y~x, kernel = "linear", cost = 10, scale=F)
plot(x, col=y, xlim=c(-3,6), ylim=c(-2.5,6.5))
table(y, predict(fit))