scikits learn SVM - 1-dimensional Separating Hyperplane

Question

How to plot the separating "hyperplane" for 1-dimensional data using scikit svm ?

I follow this guide for 2-dimensional data : http://scikit-learn.org/stable/auto_examples/svm/plot_svm_margin.html, but don't know how to make it works for 1-dimensional data

pos = np.random.randn(20, 1) + 1
neg = np.random.randn(20, 1) - 1
X = np.r_[pos, neg]
Y = [0] * 20 + [1] * 20
clf = svm.SVC(kernel='linear', C=0.05)
clf.fit(X, Y)

# how to get "hyperplane" and margins values ??

thanks

Answer 1

The separating hyperplane for two-dimensional data is a line, whereas for one-dimensional data the hyperplane boils down to a point. The easiest way to plot the separating hyperplane for one-dimensional data is a bit of a hack: the data are made two-dimensional by adding a second feature which takes the value 0 for all the samples. By doing so, the second component of the weight vector is zero, i.e. w = [w₀, 0] (see the appendix at the end of this post). As w₁ = 0 and w₁ is in the denominator of the expression that defines the slope and the y-intercept term of the separating line (see appendix), both coefficients are ∞. In this case it is convenient to solve the equation of the separating hyperplane for x, which results in x = x₀ = -b/w₀. The margin turns out to be 2/w₀ (see appendix for details).

The following script implements this approach:

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm

np.random.seed(0)
pos = np.hstack((np.random.randn(20, 1) + 1, np.zeros((20, 1))))
neg = np.hstack((np.random.randn(20, 1) - 1, np.zeros((20, 1))))
X = np.r_[pos, neg]
Y = [0] * 20 + [1] * 20

clf = svm.SVC(kernel='linear')
clf.fit(X, Y)
w = clf.coef_[0]
x_0 = -clf.intercept_[0]/w[0]
margin = w[0]

plt.figure()
x_min, x_max = np.floor(X.min()), np.ceil(X.max())
y_min, y_max = -3, 3
yy = np.linspace(y_min, y_max)
XX, YY = np.mgrid[x_min:x_max:200j, y_min:y_max:200j]
Z = clf.predict(np.c_[XX.ravel(), np.zeros(XX.size)]).reshape(XX.shape)
plt.pcolormesh(XX, YY, Z, cmap=plt.cm.Paired)
plt.plot(x_0*np.ones(shape=yy.shape), yy, 'k-')
plt.plot(x_0*np.ones(shape=yy.shape) - margin, yy, 'k--')
plt.plot(x_0*np.ones(shape=yy.shape) + margin, yy, 'k--')
plt.scatter(pos, np.zeros(shape=pos.shape), s=80, marker='o', facecolors='none')
plt.scatter(neg, np.zeros(shape=neg.shape), s=80, marker='^', facecolors='none')
plt.xlim(x_min, x_max)
plt.ylim(y_min, y_max)
plt.show()

Although the code above is self explanatory, here are some tips. X dimensions are 40 rows by 2 columns: the values in the first column are random numbers while all the elements of the second column are zeros. In the code, the weight vector w = [w₀, 0] and the intercept b are clf_coef_[0] and clf.intercept_[0], respectively, wehre clf if the object returned by sklearn.svm.SVC.

And this is the plot you get when the script is run:

For the sake of clarity I'd suggest to tweak the code above by adding/subtracting a small constant to the second feature, for example:

plt.scatter(pos, .3 + np.zeros(shape=pos.shape), ...)
plt.scatter(neg, -.3 + np.zeros(shape=neg.shape), ...)

By doing so the visualization is significantly improved since the different classes are shown without overlap.

---

Appendix

The separating hyperplane is usually expressed as

where x is a n-dimensional vector, w is the weight vector and b is the bias or intercept. For n = 2 we have w₀.x + w₁.y + b = 0. After some algebra we obtain y = -(w₀/w₁).x + (-b/w₁). It clearly emerges from this expression that the discriminant hyperplane in a 2D feature space is a line of equation y = a.x + y₀, where the slope is given by a = -w₀/w₁ and the y-intercept term is y₀ = -b/w₁. In SVM, the margin of a separating hyperplane is 2/‖w‖, which for 2D reduces to

Answer 2

the .coef_ member of clf will return the "hyperplane," which, in one dimension, is just a point. Check out this post for info on how to plot points on a numberline.