Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Drop all rows in Pandas DataFrame where value is NOT NaN

Tags:

python

pandas

I can remove all rows with nan in a column with this line:

df2 = df.dropna(subset=['columnA'])

How do I remove all rows that have values other than NaN?

like image 420
Cactus Philosopher Avatar asked Aug 17 '19 22:08

Cactus Philosopher


People also ask

How do you drop all rows with missing values in Pandas?

To drop all the rows which contain only missing values, pass the value 0 to the axis parameter and set the value how='all' .

How do you exclude NaN values from DataFrame?

By using dropna() method you can drop rows with NaN (Not a Number) and None values from pandas DataFrame. Note that by default it returns the copy of the DataFrame after removing rows. If you wanted to remove from the existing DataFrame, you should use inplace=True .

How do I delete rows in Pandas DataFrame based on condition?

Use pandas. DataFrame. drop() method to delete/remove rows with condition(s).

Does Dropna work on NaN?

You can remove missing values ( NaN ) from pandas. DataFrame , Series with dropna() . This article describes the following contents. If you want to extract rows and columns with missing values, see the following article.


3 Answers

You can do drop

df2 = df.dropna(subset=['columnA'])
df1 = df.drop(df2.index)
like image 123
BENY Avatar answered Oct 08 '22 03:10

BENY


df.loc[lambda x:x.columnA.isnull()]
like image 3
Mark Wang Avatar answered Oct 08 '22 01:10

Mark Wang


I might be missing something in the question.

Just keep the rows where value is equal to np.nan

As @rafaelc pointed out np.nan == np.nan is false.

And I was completely wrong I am leaving the answer here just to keep the comments here for anyone who comes looking.

Changing based on that.

df2 = df[df['ColumnA'] != np.nan]   # WRONG ANSWER
df1 = df[~(df['ColumnA'] != np.nan)] #WRONG ANSWER

# perform function on df1  # WRONG ANSWER

df_f = pd.concat([df1,df2])
like image 1
Sid Avatar answered Oct 08 '22 01:10

Sid