Training logistic regression using scikit learn for multi-class classification

Question

As per the scikit multiclass classification Logistic regression can be used for multi-class classification by setting multi_class=multinomial in the the constructor. But doing this gives error:

Code:

text_clf = Pipeline([('vect', TfidfVectorizer()),('clf', LogisticRegression(multi_class = 'multinomial')),])
text_clf = text_clf.fit(X_train, Y_train)

Error:

ValueError: Solver liblinear does not support a multinomial backend.

Can you tell me what is wrong here?

Note: Keeping multi_class to blank i.e. "ovr" is working fine but it fits a binary model for each classifier and I want to try mutlinomial feature also.

Answer 1

It looks like you are not providing solver & by default solver is set to 'liblinear' that does not support multi class. As per sklearn version 0.20.1, multiclass is being supported by ‘newton-cg’, ‘lbfgs’, ‘sag’, ‘saga’ not by ‘liblinear’ so change your instance creation for LogisticRegression as per following code

logReg = LogisticRegression(multi_class='multinomial', solver='newton-cg')

solver must be anything from ‘newton-cg’, ‘lbfgs’, ‘sag’, ‘saga’ but can not be left

Answer 2

From the doc:

Currently the ‘multinomial’ option is supported only by the ‘lbfgs’ and ‘newton-cg’ solvers.

So you need to explicitly set solver to 'newton-cg' or 'lbfgs', since the default solver is 'liblinear'.