Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

better way to drop nan rows in pandas

Tags:

python

pandas

On my own I found a way to drop nan rows from a pandas dataframe. Given a dataframe dat with column x which contains nan values,is there a more elegant way to do drop each row of dat which has a nan value in the x column?

dat = dat[np.logical_not(np.isnan(dat.x))] dat = dat.reset_index(drop=True) 
like image 227
kilojoules Avatar asked Apr 02 '16 08:04

kilojoules


People also ask

How do I drop NaN columns in pandas?

pandas. DataFrame. dropna() is used to drop/remove columns with NaN / None values.

How do you drop a row with null value in Python?

Drop all rows having at least one null valueDataFrame. dropna() method is your friend. When you call dropna() over the whole DataFrame without specifying any arguments (i.e. using the default behaviour) then the method will drop all rows with at least one missing value.

How to drop rows having NaN values in pandas Dataframe?

We can drop Rows having NaN Values in Pandas DataFrame by using dropna () function It is also possible to drop rows with NaN values with regard to particular columns using the following statement: With inplace set to True and subset set to a list of column names to drop all rows with NaN under those columns.

What is the use of dropna in pandas?

The pandas dropna() function is used to drop rows with missing values (NaNs) from a pandas dataframe. By default, it drops all rows with any NaNs.

How do I drop only rows with Nan in SQL?

Drop rows only if NAs are present in specific column (s) You can use dropna () such that it drops rows only if NAs are present in certain column (s). You can pass the columns to check for as a list to the subset parameter. In the above example, we drop only the rows that had column B as NaN.

How to remove NaN values from a Dataframe in Python?

As you may observe, the first, second and fourth rows now have NaN values: To drop all the rows with the NaN values, you may use df.dropna (). Here is the complete Python code to drop those rows with the NaN values: Run the code, and you’ll see only two rows without any NaN values:


1 Answers

Use dropna:

dat.dropna() 

You can pass param how to drop if all labels are nan or any of the labels are nan

dat.dropna(how='any')    #to drop if any value in the row has a nan dat.dropna(how='all')    #to drop if all values in the row are nan 

Hope that answers your question!

Edit 1: In case you want to drop rows containing nan values only from particular column(s), as suggested by J. Doe in his answer below, you can use the following:

dat.dropna(subset=[col_list])  # col_list is a list of column names to consider for nan values. 
like image 175
TerminalWitchcraft Avatar answered Oct 21 '22 21:10

TerminalWitchcraft