Menu
To remove NaN from a list using Python, the easiest way is to use the isnan() function from the Python math module and list comprehension. You can also use the Python filter() function. The Python numpy module also provides an isnan() function that we can use to check if a value is NaN.
Nan implies “not a number” in python language. It is usually a float-type value that does not exist in data. Due to this reason, data users must remove “nan” values. There are numerous approaches available to remove “nan” values from a list data structure.
isnan(a)) # Use a mask to mark the NaNs a_norm = a / np. sum(a) # The sum function ignores the masked values. a_norm2 = a / np. std(a) # The std function ignores the masked values.
The question has changed, so to has the answer:
Strings can't be tested using math.isnan as this expects a float argument. In your countries list, you have floats and strings.
In your case the following should suffice:
cleanedList = [x for x in countries if str(x) != 'nan']
In your countries list, the literal 'nan' is a string not the Python float nan which is equivalent to:
float('NaN')
In your case the following should suffice:
cleanedList = [x for x in countries if x != 'nan']
Using your example where...
countries= [nan, 'USA', 'UK', 'France']
Since nan is not equal to nan (nan != nan) and countries[0] = nan, you should observe the following:
countries[0] == countries[0]
False
However,
countries[1] == countries[1]
True
countries[2] == countries[2]
True
countries[3] == countries[3]
True
Therefore, the following should work:
cleanedList = [x for x in countries if x == x]
The problem comes from the fact that np.isnan() does not handle string values correctly. For example, if you do:
np.isnan("A")
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
However the pandas version pd.isnull() works for numeric and string values:
pd.isnull("A")
> False
pd.isnull(3)
> False
pd.isnull(np.nan)
> True
pd.isnull(None)
> True
import numpy as np
mylist = [3, 4, 5, np.nan]
l = [x for x in mylist if ~np.isnan(x)]
This should remove all NaN. Of course, I assume that it is not a string here but actual NaN (np.nan).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With