Using a list of conditions to filter a DataFrame in Pandas

Question

I wish to have a function which takes a list of conditions, of any length, and places an ampersand between all the conditions. Example code below.

df = pd.DataFrame(columns=['Sample', 'DP','GQ', 'AB'],
         data=[
               ['HG_12_34', 200, 35, 0.4],
               ['HG_12_34_2', 50, 45, 0.9],
               ['KD_89_9', 76, 67, 0.7],
               ['KD_98_9_2', 4, 78, 0.02],
               ['LG_3_45', 90, 3, 0.8],
               ['LG_3_45_2', 15, 12, 0.9]
               ])


def some_func(df, cond_list):

    # wrap ampersand between multiple conditions
    all_conds = ?

    return df[all_conds]

cond1 = df['DP'] > 40
cond2 = df['GQ'] > 40
cond3 = df['AB'] < 0.4


some_func(df, [cond1, cond2]) # should return df[cond1 & cond2]
some_func(df, [cond1, cond3, cond2]) # should return df[cond1 & cond3 & cond2]

I would appreciate any help with this.

Answer 1

You can use functools.reduce for that:

from functools import reduce

def some_func(df, cond_list):
    return df[reduce(lambda x,y: x&y, cond_list)]

Or, like @AryaMcCarthy says, you can use and_ from the operator package:

from functools import reduce
from operator import and_

def some_func(df, cond_list):
    return df[reduce(and_, cond_list)]

or with numpy - like @ayhan says - which has also a logical and reduction:

from numpy import logical_and

def some_func(df, cond_list):
    return df[logical_and.reduce(cond_list)]

All three versions produce - for your sample input - the following output:

>>> some_func(df, [cond1, cond2])
       Sample  DP  GQ   AB
1  HG_12_34_2  50  45  0.9
2     KD_89_9  76  67  0.7
>>> some_func(df, [cond1, cond2, cond3])
Empty DataFrame
Columns: [Sample, DP, GQ, AB]
Index: []