Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pandas .dropna() on specify attribute

Tags:

python

pandas

I have this code to drop null values from column Type, specifically looking at Dog.

cd.loc[cd['Type'] == 'Dog'].dropna(subset = ['Killed'], inplace = True)

I would like to dropna when the ['Killed'] column associating with Type = Dog has NaN value.

The code above generate this pandas error:

 A value is trying to be set on a copy of a slice from a DataFrame

Is there another way where can I dropna on ['Killed'] when ['Type'] == 'Dog'?

(This is my first post), sorry if I can't explain properly Cheers

like image 469
Phurich.P Avatar asked Aug 31 '16 05:08

Phurich.P


People also ask

How do you Dropna row in pandas?

To drop all the rows with the NaN values, you may use df. dropna().

How do I drop a specific value in pandas?

We can use the column_name function along with the operator to drop the specific value.

Does Dropna drop the entire row?

Pandas Drop All Rows with any Null/NaN/NaT Values This is the default behavior of dropna() function.


2 Answers

Very similar to @BrenBarn's answer but using drop and inplace

cd.drop(cd[(cd.Type == 'Dog') & (cd.Killed.isnull())].index, inplace=True)

Setup

cd = pd.DataFrame([
        ['Dog', 'Yorkie'],
        ['Cat', 'Rag Doll'],
        ['Cat', None],
        ['Bird', 'Caique'],
        ['Dog', None],
    ], columns=['Type', 'Killed'])

Solution

cd.drop(cd[(cd.Type == 'Dog') & (cd.Killed.isnull())].index, inplace=True)

cd

enter image description here


Equivalently with DeMorgan's law

cond1 = cd.Type == 'Dog'
cond2 = cd.Killed.isnull()
cd[~cond1 | ~cond2]

A silly one, because I felt like it!

cd.groupby('Type', group_keys=False) \
    .apply(lambda df: df.dropna(subset=['Killed']) if df.name == 'Dog' else df)
like image 167
piRSquared Avatar answered Sep 18 '22 08:09

piRSquared


It sounds like what you are saying is you want to remove rows where Type is "Dog" and Killed is NaN. So just select the negation of that condition:

cd = cd.loc[~((cd.Type=="Dog") & cd.Killed.isnull())]
like image 28
BrenBarn Avatar answered Sep 20 '22 08:09

BrenBarn