How to get all alpha values of scikit-learn SVM classifier?

Question

I need the alpha values, which are the Lagrange multipliers of the SVM dual problem, after training a SVM classifier with scikit-learn. According to the document, it seems that scikit-learn provides only svm.dual_coef_, which is the product of the Lagrange multiplier alpha and the label of a data point.

I tried to calculate the alpha value manually by dividing the elements of svm.dual_coef_ by the data label, but since svm.dual_coef_ stores only the coefficients of the support vectors, I'm not sure if I iterate over this array, the order of support vectors would be the same as the order in the original training data.

So is there a reliable way to get the alpha values of support vectors?

Answer 1

As alpha values are by definition positive you can get it through taking abs of dual_coefs:

alphas = np.abs(svm.dual_coef_)

whis is a direct consequence of the fact that

svm.dual_coef_[i] = labels[i] * alphas[i]

where labels[i] is either -1 or +1 and alphas[i] are always positive. Futhermore, you can also get each label through

labels = np.sign(svm.dual_coef_)

using the same observation. This is also why scikit-learn does not store alphas as such - they are uniquely represented by dual_coefs_, together with labels.

It is easy to understand it once you analyze all possible cases:

• labels[i] == -1 and alphas[i] > 0 => dual_coef_[i] < 0 and dual_coef_[i] == -alphas[i] == labels[i] * alphas[i]
• labels[i] == -1 and alphas[i] < 0 => impossible (alphas are non-negative)
• labels[i] == -1 and alphas[i]== 0 => it is not a support vector
• labels[i] == +1 and alphas[i] > 0 => dual_coef_[i] > 0 and dual_coef_[i] == alphas[i] == labels[i] * alphas[i]
• labels[i] == +1 and alphas[i] < 0 => impossible (alphas are non-negative)
• labels[i] == +1 and alphas[i]== 0 => it is not a support vector

Consequently, if dual_coef_[i] is positive then it is the alphas[i] coefficient, and it belongs to positive class, and if it is negative, alphas[i] == -dual_coef_[i] and it belongs to negative class.