Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python Pandas - How to filter multiple columns by one value

I am doing analysis by pandas.
My table has 7M rows* 30 columns. Cell values are ranged from -1 to 3 randomly. Now I want to filter out rows based on columns' value.

I understand how to select based on multiple conditions, write down conditions and combine by "&" "|".
But I have 30 columns to filter and filter by the same value. For instance, last 12 columns' value equals -1 need to be selected

df.iloc[:,-12:]==-1

The code above gives me a boolean. I need actual data frame.
The logic here is "or", means if any column has value -1, that row needs to be selected. Also, it is good to know what if I need "and", all columns have value -1?
Many thanks

like image 344
Baiii Avatar asked Mar 10 '17 05:03

Baiii


1 Answers

For the OR case, use DF.any (returns True if any element is True along a particular axis):

df[(df.iloc[:,-12:] == -1).any(axis=1)]

For the AND case, use DF.all (returns True if all elements are True along a particular axis):

df[(df.iloc[:,-12:] == -1).all(axis=1)]
like image 118
Nickil Maveli Avatar answered Sep 26 '22 03:09

Nickil Maveli