How to disable ConvergenceWarning using sklearn?

Question

I'm using GridSearchCV to optimize hyper-parameters for SVM. I set the maximum number of iterations because I can't wait several hours to get result. I know there will be convergence warnings. I just want to ignore these warnings and not show up in the terminal.

Thanks in advance.

Answer 1

This was a pain to track down, as all suggested answers I've seen simply do not work. What finally worked for me was in the example code Early stopping of Stochastic Gradient Descent:

from sklearn.utils.testing import ignore_warnings
from sklearn.exceptions import ConvergenceWarning

You can then annotate a function like so:

@ignore_warnings(category=ConvergenceWarning)
def my_function():
    # Code that triggers the warning

Note that you need not directly import anything from warnings.

I think that's quite nice as it will only suppress warnings in the specific case where you need it, rather than globally.

Answer 2

I will take a long shot here.

You have not provided enough information. You just mentioned that you're using SVM but not which type of SVM since there are many implementations of it such as SVC, NuSVC and LinearSVC. Those different types have different properties.

Why to care? because some of them support/accept executing jobs in parallel such as LinearSVC one!

with warnings.catch_warnings():
    warnings.filterwarnings("ignore", category=ConvergenceWarning)

The above code (or the other variants of it) should do the job, but if it is running in parallel, it will only do in the very first run/iteration ( I am not pretty sure why but it seems every job has its own Pythonic configuration as if it is a new instance or something!)

---

Also, you mentioned that you are using GridSearchCV which has n_job parameter as well. Its Scikit Documentation says:

Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors

joblib.parallel_backend means what number of jobs is set in the estimator or any per-defined configurations.

---

Summary

Running jobs in parallel can be the reason of not suppressing the warnings. More information from OP is required.

---

EDIT

I checked it again, and indeed, using GridSearchCV with scikit-learn version 0.20.3 and low max_iter while suppressing warnings, lead to the following results:

1. SVC or LinearSVC + GridSearchCV(n_jobs=-1 or >1): Failed to suppress warnings.
2. SVC or LinearSVC + GridSearchCV(n_jobs=None or 1): Succeeded in suppressing warnings.
3. LogisticRegression(n_jobs=-1, solver='sag') + GridSearchCV(n_jobs=None or 1 or >1 or -1): Failed to suppress warnings.
4. LogisticRegression(n_jobs=1, solver='sag') + GridSearchCV(n_jobs=-1 or >1): Failed to suppress warnings.
5. LogisticRegression(n_jobs=1, solver='sag') + GridSearchCV(n_jobs=None or 1): Succeeded in suppressing warnings.

As you can see, if the estimator supports multi-jobs, setting n_jobs=-1 or >1 will not suppress warnings regardless of n_jobs in GridSearchCV. On the other hand, if the estimator does not support multi-jobs, setting n_jobs=-1 or >1 in GridSearchCV will not make warnings suppression work, however, setting n_jobs=None or 1 will do make it work.

Important Note

That is what I found with scikit-learn version 0.20.3, nevertheless, I tried it on my other laptop with scikit-learn version 0.19.2 and suppressing warnings worked all times regardless! I checked scikit-learn GitHub repository and noticed some commits about joblib since version 0.19.2 but I am not sure if there was a real change/update that caused the above behavior! You may want to open a ticket there and refer to the above results.

---

UPDATE

The only way I could suppress all Scikit-learn warnings, is by issuing the following code at the beginning of the module. (but note that will suppress all warnings including yours - I needed that because I have logs saved to database):

if not sys.warnoptions:
    warnings.simplefilter("ignore")
    os.environ["PYTHONWARNINGS"] = "ignore" # Also affect subprocesses

Answer 3

In order to control Python warnings you can use the warnings library. See detailed documentation here. So you can use warning.simplefilter() method as follows:

from warnings import simplefilter
from sklearn.exceptions import ConvergenceWarning
simplefilter("ignore", category=ConvergenceWarning)