Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Efficiently drop dataframe rows where *index* contains nulls (NaT)

My data files infrequently contain malformed lines (think abrupt power loss). When badness occurs across a timestamp, making it uninterpretable, the resulting DataFrame.Index contains Not-a-Time (NaT) values (because I've coerced it to).

My real problem is that instances of NaT prevent the use of resample. I need to remove them, first. Unfortunately, I haven't figured out if/how to use dropna on the index itself. It's looking more and more like I need to make the index a column, operate on it, then re-make it the index. (But I don't want to do that.)

Is there an established idiom for dropping dataframe rows where the Index values are null?

like image 851
patricktokeeffe Avatar asked Apr 28 '16 19:04

patricktokeeffe


People also ask

How do you drop a row with null value in Python?

Drop all rows having at least one null value When it comes to dropping null values in pandas DataFrames, pandas. DataFrame. dropna() method is your friend.

How do I drop missing values in pandas Python?

To drop all the rows which contain only missing values, pass the value 0 to the axis parameter and set the value how='all' .


1 Answers

Use

df.loc[pd.notnull(df.index)]

for newer versions you can do

df.loc[df.index.notnull()]

or

df.loc[df.index.dropna()]
like image 178
EdChum Avatar answered Sep 30 '22 03:09

EdChum