ValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty

Question

I'm running the process of feature selection on classification problem, using the embedded method (L1 - Lasso) With LogisticRegression.

I'm running the following code:

from sklearn.linear_model import Lasso, LogisticRegression
from sklearn.feature_selection import SelectFromModel

# using logistic regression with penalty l1.
selection = SelectFromModel(LogisticRegression(C=1, penalty='l1'))
selection.fit(x_train, y_train)

But I'm getting exception (on the fit command):

   selection.fit(x_train, y_train)
   File "C:\Python37\lib\site-packages\sklearn\feature_selection\_from_model.py", line 222, in fit
   self.estimator_.fit(X, y, **fit_params)
   File "C:\Python37\lib\site-packages\sklearn\linear_model\_logistic.py", line 1488, in fit
   solver = _check_solver(self.solver, self.penalty, self.dual)
   File "C:\Python37\lib\site-packages\sklearn\linear_model\_logistic.py", line 445, in _check_solver
   "got %s penalty." % (solver, penalty))
   ValueError: Solver lbfgs supports only 'l2' or 'none' penalties, got l1 penalty.

I'm running under python 3.7.6 and sscikit-learn version is 0.22.2.post1

What is wrong and how can I fix it ?

Answer 1

This is cleared up in the documentation.

solver : {‘newton-cg’, ‘lbfgs’, ‘liblinear’, ‘sag’, ‘saga’}, default=’lbfgs’

...

• ‘newton-cg’, ‘lbfgs’, ‘sag’ and ‘saga’ handle L2 or no penalty

• ‘liblinear’ and ‘saga’ also handle L1 penalty

Call it like this:

LogisticRegression(C=1, penalty='l1', solver='liblinear')

Answer 2

As l1 is supported by solver 'liblinear'. Always specify solver='liblinear' with penalty= 'l1'

selection = SelectFromModel(LogisticRegression(C=1, penalty='l1', solver='liblinear'))

Answer 3

Just try to specify the solver that you want to use and then the error will be gone. l1 support 'liblinear' and 'saga' L2 handle newton-cg’, ‘lbfgs’, ‘sag’ and ‘saga’

clf = LogisticRegression(C=0.01, penalty='l1',solver='liblinear');