Which decision_function_shape for sklearn.svm.SVC when using OneVsRestClassifier?

Question

I am doing multi-label classification where I am trying to predict correct tags to questions:

(X = questions, y = list of tags for each question from X).

I am wondering, which decision_function_shape for sklearn.svm.SVC should be be used with OneVsRestClassifier?

From docs we can read that decision_function_shape can have two values 'ovo' and 'ovr':

decision_function_shape : ‘ovo’, ‘ovr’ or None, default=None

Whether to return a one-vs-rest (‘ovr’) decision function of shape (n_samples, n_classes) as all other classifiers, or the original one-vs-one (‘ovo’) decision function of libsvm which has shape (n_samples, n_classes * (n_classes - 1) / 2). The default of None will currently behave as ‘ovo’ for backward compatibility and raise a deprecation warning, but will change ‘ovr’ in 0.19.

But I still don't understand what is the difference between:

# First decision_function_shape set to 'ovo'
estim = OneVsRestClassifier(SVC(kernel='linear', decision_function_shape ='ovo'))

# Second decision_function_shape set to 'ovr'
estim = OneVsRestClassifier(SVC(kernel='linear', decision_function_shape ='ovr'))

Which decision_function_shape should be used for multi-label classification problem?

EDIT: Question asking a similar thing with no answer.

Answer 1

I think the question of which should be used is best left up to a situational. That could easily be a part of your GridSearch. But just intuitively I would feel that as far as differences go you are going to be doing the same thing. Here is my reasoning:

OneVsRestClassifier is designed to model each class against all of the other classes independently, and create a classifier for each situation. The way I understand this process is that OneVsRestClassifier grabs a class, and creates a binary label for whether a point is or isn't that class. Then this labelling gets fed into whatever estimator you have chosen to use. I believe the confusion comes in in that SVC also allows you to make this same choice, but in effect with this implementation the choice will not matter because you will always only be feeding two classes into the SVC.

And here is an example:

from sklearn.datasets import load_iris
from sklearn.multiclass import OneVsRestClassifier
from sklearn.svm import SVC

data = load_iris()

X, y = data.data, data.target
estim1 = OneVsRestClassifier(SVC(kernel='linear', decision_function_shape='ovo'))
estim1.fit(X,y)

estim2 = OneVsRestClassifier(SVC(kernel='linear', decision_function_shape='ovr'))
estim2.fit(X,y)

print(estim1.coef_ == estim2.coef_)
array([[ True,  True,  True,  True],
       [ True,  True,  True,  True],
       [ True,  True,  True,  True]], dtype=bool)

So you can see the coefficients are all equal for all three estimators built by the two models. Granted this dataset only has 150 samples and 3 classes so it is possible these results could be different for a more complex dataset, but it's a simple proof of concept.