Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Numpy isnan() fails on an array of floats (from pandas dataframe apply)

People also ask

Is numpy NaN float?

NaN stands for Not A Number and is a common missing data representation. It is a special floating-point value and cannot be converted to any other type than float.

What is NP Isnan in Python?

The numpy.isnan() function tests element-wise whether it is NaN or not and returns the result as a boolean array.

Why is NaN a float pandas?

In data science, Nan is used to represent the missing values in a dataset. So Nan is basically a placeholder to represent undefined or missing values. You can create the Nan value using float type. Since it is not a defined keyword in Python, you have to pass it to float in a string format (within quotes).

Is NaN check numpy?

To check for NaN values in a Numpy array you can use the np. isnan() method. This outputs a boolean mask of the size that of the original array. The output array has true for the indices which are NaNs in the original array and false for the rest.


np.isnan can be applied to NumPy arrays of native dtype (such as np.float64):

In [99]: np.isnan(np.array([np.nan, 0], dtype=np.float64))
Out[99]: array([ True, False], dtype=bool)

but raises TypeError when applied to object arrays:

In [96]: np.isnan(np.array([np.nan, 0], dtype=object))
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Since you have Pandas, you could use pd.isnull instead -- it can accept NumPy arrays of object or native dtypes:

In [97]: pd.isnull(np.array([np.nan, 0], dtype=float))
Out[97]: array([ True, False], dtype=bool)

In [98]: pd.isnull(np.array([np.nan, 0], dtype=object))
Out[98]: array([ True, False], dtype=bool)

Note that None is also considered a null value in object arrays.


A great substitute for np.isnan() and pd.isnull() is

for i in range(0,a.shape[0]):
    if(a[i]!=a[i]):
       //do something here
       //a[i] is nan

since only nan is not equal to itself.


On top of @unutbu answer, you could coerce pandas numpy object array to native (float64) type, something along the line

import pandas as pd
pd.to_numeric(df['tester'], errors='coerce')

Specify errors='coerce' to force strings that can't be parsed to a numeric value to become NaN. Column type would be dtype: float64, and then isnan check should work


Make sure you import csv file using Pandas

import pandas as pd

condition = pd.isnull(data[i][j])