sklearn DeprecationWarning truth value of an array

Question

Running a rasa_core example from the docs with

› python3 -m rasa_core.run -d models/dialogue -u models/nlu/default/current

and get this error output after each message in the dialog:

.../sklearn/...: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use `array.size > 0` to check that an array is not empty.

It's an issue with numpy that has been fixed but not been published in the latest release: https://github.com/scikit-learn/scikit-learn/issues/10449

The following has not worked to temporarily silence the warning:

1. Adding -W ignore

python3 -W ignore -m rasa_core.run -d models/dialogue -u models/nlu/default/current

2. warnings.simplefilter

python3

>>> warnings.simplefilter('ignore', DeprecationWarning)
>>> exit()

python3 -m rasa_core.run -d models/dialogue -u models/nlu/default/current

Answer 1

This warning is caused by numpy which deprecated the truth value check on empty array

Rationale for this change is

It is impossible to take advantage of the fact that empty arrays are False, because an array can be False for other reasons.

Check following example:

>>> import numpy as np
>>> bool(np.array([]))
False
>>> # but this is not a good way to test for emptiness, because...
>>> bool(np.array([0]))
False

Solution

As per issue 10449 on scikit-learn library, this has been fixed in master branch of library. However that will be available around August 2018 so one possible alternate is to use a lesser version of numpy library which does not have this issue i.e. 1.13.3 since scikit-library by default would refer latest version of numpy(which is 1.14.2 at the time of writing this answer)

sudo pip install numpy==1.13.3

or with pip3 as follows

sudo pip3 install numpy==1.13.3

Ignoring the warning(s)

In case we want to use the latest version of library(numpy in this case) which is giving the deprecation warning and just want to silence the deprecation warning then we can achieve it by using filterwarnings method of python's Warnings module

Following example below would produce the deprecation warning mentioned in question above:

from sklearn import preprocessing

if __name__ == '__main__':
    le = preprocessing.LabelEncoder()
    le.fit([1, 2, 2, 6])
    le.transform([1, 1, 2, 6])
    le.inverse_transform([0, 0, 1, 2])

produces

/usr/local/lib/python2.7/dist-packages/sklearn/preprocessing/label.py:151: DeprecationWarning: The truth value of an empty array is ambiguous. Returning False, but in future this will result in an error. Use array.size > 0 to check that an array is not empty.

And to take care of it, add filterwarnings for DeprecationWarning

from sklearn import preprocessing
import warnings

if __name__ == '__main__':
    warnings.filterwarnings(action='ignore', category=DeprecationWarning)
    le = preprocessing.LabelEncoder()
    le.fit([1, 2, 2, 6])
    le.transform([1, 1, 2, 6])
    le.inverse_transform([0, 0, 1, 2])

In case there are multiple modules which are giving warning and we want to selectively silent warning then use module attribute. e.g. to silent deprecation warning from scikit learn module

warnings.filterwarnings(module='sklearn*', action='ignore', category=DeprecationWarning)