Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Finding missing values in a numpy array

Tags:

python

numpy

Alright, extreme rookie question here. In my program, I generate a 2D numpy array, some of whom's entries are missing (not the "nan" kind of nonexistant, but the "None" kind, or NoneType). I'd like to put a mask over these entries, but I seem to be having some trouble doing so. Ordinarily, to mask over, say, all entries with value 2, I'd do

A = np.ma.masked_where(A[A==2], A)

In this case, that doesn't seem to work no matter what I try for the first parameter. Thoughts?

like image 772
user391045 Avatar asked Jul 16 '10 06:07

user391045


People also ask

How do I find missing values in numpy?

Droping the missing values or nan values can be done by using the function "numpy. isnan()" it will give us the indexes which are having nan values and when combined with other function which is "numpy. logical_not()" where the boolean values will be reversed.

What is NaN in numpy array?

Introduction to NumPy NaN. In Python, NumPy NAN stands for not a number and is defined as a substitute for declaring value which are numerical values that are missing values in an array as NumPy is used to deal with arrays in Python and this can be initialized using numpy.


1 Answers

Since you have -- entries in your array, I guess that it means that they are already masked:

>>> m = ma.masked_where([True, False]*5, arange(10))
>>> print m
[-- 1 -- 3 -- 5 -- 7 -- 9]

So, I would say that your entries are already masked and that you can directly use your array.

If you want to create an array that only contains the non-masked value, you can do

>>> m[~m.mask]
[1 3 5 7]

where m is your masked array.

If you want to have the list of masked values, you can simply select the other values:

>>> m[m.mask]
[0 2 4 6 8]

Note that the missing values are not None, but are the original values, generally. In fact, an array of integers cannot contain None.

If you want the indices of the masked values, you can do:

>>> numpy.nonzero(m.mask)

The documentation of numpy.nonzero() describes how its result must be interpreted.

like image 191
Eric O Lebigot Avatar answered Nov 11 '22 11:11

Eric O Lebigot