Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Trying to drop NaN indexed row in dataframe

I'm using python 2.7.3 and Pandas version 0.12.0.

I want to drop the row with the NaN index so that I only have valid site_id values.

print df.head()             special_name site_id NaN          Banana OMG          Apple  df.drop(df.index[0])  TypeError: 'NoneType' object is not iterable 

If I try dropping a range, like this:

df.drop(df.index[0:1]) 

I get this error:

AttributeError: 'DataFrame' object has no attribute 'special_name' 
like image 728
Alison S Avatar asked Oct 29 '13 23:10

Alison S


People also ask

How do I delete a NaN row?

To drop all the rows with the NaN values, you may use df. dropna().

How do I drop a row based on an index?

To drop rows based on certain conditions, select the index of the rows which pass the specific condition and pass that index to the drop() method. In this code, (df['Unit_Price'] >400) & (df['Unit_Price'] < 600) is the condition to drop the rows.

How do I drop a NaN row in a DataFrame?

By using dropna() method you can drop rows with NaN (Not a Number) and None values from pandas DataFrame. Note that by default it returns the copy of the DataFrame after removing rows. If you wanted to remove from the existing DataFrame, you should use inplace=True .


1 Answers

With pandas version >= 0.20.0 you can:

df = df[df.index.notnull()] 

With older versions:

df = df[pandas.notnull(df.index)] 

To break it down:

notnull generates a boolean mask, e.g. [False, False, True], where True denotes the value at the corresponding position is null (numpy.nan or None). We then select the rows whose index corresponds to a true value in the mask by using df[boolean_mask].

like image 125
Tim Diels Avatar answered Sep 22 '22 16:09

Tim Diels