Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get rid of NaT values from pandas dataframe

Tags:

python

pandas

I have a dataframe that looks like shown below

                               mean
comp_name  date                      
Appdynamics 2012-05-01 00:18:15.910000
            2012-05-01             NaT
            2012-05-01             NaT
            2012-05-02 00:20:12.145200
            2012-05-02             NaT
            2012-05-02             NaT

Here the comp_name and date form multiindex. I want to get rid of the NaT values and obtain only those rows where the mean(timedelta64) is not NaT.

                               mean
comp_name  date                      
Appdynamics 2012-05-01 00:18:15.910000
            2012-05-02 00:20:12.145200

Any ideas on this?

like image 200
user3527975 Avatar asked Jul 03 '14 06:07

user3527975


People also ask

How do I drop NaN values?

By using dropna() method you can drop rows with NaN (Not a Number) and None values from pandas DataFrame. Note that by default it returns the copy of the DataFrame after removing rows. If you wanted to remove from the existing DataFrame, you should use inplace=True .

How do I remove a value from Pandas?

Use drop() method to delete rows based on column value in pandas DataFrame, as part of the data cleansing, you would be required to drop rows from the DataFrame when a column value matches with a static value or on another column value.

How do I find my NaT value in Pandas?

Use df. isna() to find NaN , NaT , and None values. They all evaluate to True with this method. A boolean DataFrame is returned if df.

How do you remove NaN values from a column in Python?

dropna() to drop columns having Nan values.


1 Answers

pandas.notnull() takes a series and returns a Boolean series which is True where the input series is not null (None, np.NaN, np.NaT). Then you can slice a dataframe by the Boolean series:

df[pandas.notnull(df['mean'])]
like image 94
exp1orer Avatar answered Sep 23 '22 09:09

exp1orer