What does the CV stand for in sklearn.linear_model.LogisticRegressionCV?

Question

scikit-learn has two logistic regression functions:

• sklearn.linear_model.LogisticRegression
• sklearn.linear_model.LogisticRegressionCV

I'm just curious what the CV stands for in the second one. The only acronym I know in ML that matches "CV" is cross-validation, but I'm guessing that's not it, since that would be achieved in scikit-learn with a wrapper function, not as part of the logistic regression function itself (I think).

Answer 1

You are right in guessing that the latter allows the user to perform cross validation. The user can pass the number of folds as an argument cv of the function to perform k-fold cross-validation (default is 10 folds with StratifiedKFold).

I would recommend reading the documentation for the functions LogisticRegression and LogisticRegressionCV

Answer 2

Yes, it's cross-validation. Excerpt from the docs:

For the grid of Cs values (that are set by default to be ten values in a logarithmic scale between 1e-4 and 1e4), the best hyperparameter is selected by the cross-validator StratifiedKFold, but it can be changed using the cv parameter.

The point here is the following:

• yes: sklearn has general model-selection wrappers providing CV-functionality for all those classifiers/regressors
• but: when the classifier/regressor is known/fixed a-priori (to some extent) or sometimes even some CV-model, one can gain advantages using these facts with specialized code bound to one classifier/regressor resulting in improved performance!
  • Typically:
    • CV already embedded in optimization-algorithm
    • Efficient warm-starting (instead of full re-optimization after just the change of one parameter like alpha)

It seems, at least the latter idea is used in sklearn's LogisticRegressionCV, as seen in this excerpt:

In the case of newton-cg and lbfgs solvers, we warm start along the path i.e guess the initial coefficients of the present fit to be the coefficients got after convergence in the previous fit, so it is supposed to be faster for high-dimensional dense data.