Why does "None in numpy.asarray(...)" cause a future warning

Question

The future warning happens when you do something like this:

>>> numpy.asarray([1,2,3,None]) == None

Which currently returns False, but I understand will return an array containing [False,False,False,True] in a future version of Numpy.

As discussed on the numpy discussion list the way around this is to testa is None.

What confuses me is this behaviour of the in keyword with a 1D array compared to a list:

>>> None in [1,2,3,None]
True
>>> None in numpy.asarray([1,2,3,None])
__main__:1: FutureWarning: comparison to 'None' will result in an elementwise 
    object comparison in the future
False
>>> 1 in numpy.asarray([1,2,3,None])
True

EDIT (see comments) - There are really two different questions:

1. Why does this cause a FutureWarning - what will the future behaviour of None in numpy.asarray(...) be compared to what it is now?
2. Why the difference in behaviour of in from a list; can I test if my array contains None without converting it to a list or using a for loop?

Numpy version is 1.9.1, Python 3.4.1

Answer 1

The future warning happens when you do something like this:

numpy.asarray([1,2,3,4]) == None

Which currently returns False, but I understand will return an array containing [False,False,False,True] in a future version of Numpy.

As I mentioned in the comments, your example is incorrect. Future versions of numpy would return [False ,False, False, False], i.e. False for each element in the array that is not equal to None. This is more consistent with how element-wise comparisons to other scalar values currently work, e.g.:

In [1]: np.array([1, 2, 3, 4]) == 1
Out[1]: array([ True, False, False, False], dtype=bool)

In [2]: np.array(['a', 'b', 'c', 'd']) == 'b'
Out[2]: array([False,  True, False, False], dtype=bool)

What confuses me is this behaviour of the in keyword with a 1D array compared to a list

When you test x in y, you are calling y.__contains__(x). When y is a list, __contains__ basically does something along the lines of this:

for item in y:
    if (item is x) or (item == x):
        return True
return False

As far as I can tell, np.ndarray.__contains__(x) performs the equivalent of this:

if any(y == x):
    return True
else:
    return False

That is to say it tests element-wise equality over the whole array first (y == x would be a boolean array the size of y). Since in your case you are testing whether y == None, this will raise the FutureWarning for the reasons given above.

In the comments you also wanted to know why

np.nan in np.array([1, 2, 3, np.nan])

returns False, but

np.nan in [1, 2, 3, np.nan]

returns True. The first part is easily explained by the fact that np.nan != np.nan (see here for the rationale behind this). To understand why the second case returns True, remember that list.__contains__() first checks for identity (is) before checking equality (==). Since np.nan is np.nan, the second case will return True.