Remove all missing values ( NaN )By using the negation operator ~ for this ndarray and setting NaN to False , the missing values can be deleted (= non-missing values are extracted). Since the number of remaining elements is different, the shape of the original array is not maintained and is flattened. print(~np.
Method 1: By using rmmissing( ) This function is used to remove missing entries or Nan values from a specified matrix.
You can use numpy. nan_to_num : numpy. nan_to_num(x) : Replace nan with zero and inf with finite numbers.
If you're using numpy for your arrays, you can also use
x = x[numpy.logical_not(numpy.isnan(x))]
Equivalently
x = x[~numpy.isnan(x)]
[Thanks to chbrown for the added shorthand]
Explanation
The inner function, numpy.isnan
returns a boolean/logical array which has the value True
everywhere that x
is not-a-number. As we want the opposite, we use the logical-not operator, ~
to get an array with True
s everywhere that x
is a valid number.
Lastly we use this logical array to index into the original array x
, to retrieve just the non-NaN values.
filter(lambda v: v==v, x)
works both for lists and numpy array since v!=v only for NaN
Try this:
import math
print [value for value in x if not math.isnan(value)]
For more, read on List Comprehensions.
For me the answer by @jmetz didn't work, however using pandas isnull() did.
x = x[~pd.isnull(x)]
As shown by others
x[~numpy.isnan(x)]
works. But it will throw an error if the numpy dtype is not a native data type, for example if it is object. In that case you can use pandas.
x[~pandas.isna(x)] or x[~pandas.isnull(x)]
@jmetz's answer is probably the one most people need; however it yields a one-dimensional array, e.g. making it unusable to remove entire rows or columns in matrices.
To do so, one should reduce the logical array to one dimension, then index the target array. For instance, the following will remove rows which have at least one NaN value:
x = x[~numpy.isnan(x).any(axis=1)]
See more detail here.
Doing the above :
x = x[~numpy.isnan(x)]
or
x = x[numpy.logical_not(numpy.isnan(x))]
I found that resetting to the same variable (x) did not remove the actual nan values and had to use a different variable. Setting it to a different variable removed the nans. e.g.
y = x[~numpy.isnan(x)]
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With