Python Numpy mask NaN not working

Question

I'm simply trying to use a masked array to filter out some nanentries.

import numpy as np
# x = [nan, -0.35, nan]
x = np.ma.masked_equal(x, np.nan)
print x

This outputs the following:

masked_array(data = [        nan -0.33557216         nan],
         mask = False,
   fill_value = nan)

Calling np.isnan() on x returns the correct boolean array, but the mask just doesn't seem to work. Why would my mask not be working as I expect?

Answer 1

You can use np.ma.masked_invalid:

import numpy as np

x = [np.nan, 3.14, np.nan]
mx = np.ma.masked_invalid(x)

print(repr(mx))
# masked_array(data = [-- 3.14 --],
#              mask = [ True False  True],
#        fill_value = 1e+20)

Alternatively, use np.isnan(x) as the mask= parameter to np.ma.masked_array:

print(repr(np.ma.masked_array(x, np.isnan(x))))
# masked_array(data = [-- 3.14 --],
#              mask = [ True False  True],
#        fill_value = 1e+20)

Why doesn't your original approach work? Because, rather counterintuitively, NaN is not equal to NaN!

print(np.nan == np.nan)
# False

This is actually part of the IEEE-754 definition of NaN

Answer 2

Here is another alternative without using mask:

import numpy as np
#x = [nan, -0.35, nan]
xmask=x[np.logical_not(np.isnan(x))]
print(xmask)

Result:

array([-0.35])