Filtering a Dataframe using dictionary with multiple elements

Question

I've tried for a few hours now to find an answer here but I am unable to get any to work in my particular case. Closest I could find was this: Apply multiple string containment filters to pandas dataframe using dictionary

I have a pd.Dataframe of deal prices with the following columns:

df1 = database[['DealID',
         'Price',
         'Attribute A',
         'Attribute B',
         'Attribute C']]

The attributes are categorised into the following:

filter_options = {
    'Attribute A': ["A1","A2","A3","A4"],
    'Attribute B': ["B1","B2","B3","B4"],
    'Attribute C': ["C1","C2","C3"],
}

I want to filter df1 using a subset of filter_options which has multiple values per key:

filter = {
    'Attribute A': ["A1","A2"],
    'Attribute B': ["B1"],
    'Attribute C': ["C1","C3"],
}

The below works fine when there is only one value per key in the dictionary.

df_filtered = df1.loc[(df1[list(filter)] == pd.Series(filter)).all(axis=1)]

However, am i able to get the same outcome with multple values per key?

Thanks!

Answer 1

I believe you need change variable filter because python reserved word and then use list comprehension with isin and concat for boolean mask:

df1 = pd.DataFrame({'Attribute A':["A1","A2"],
                    'Attribute B':["B1","B2"],
                    'Attribute C':["C1","C2"],
                    'Price':[140,250]})

filt = {
    'Attribute A': ["A1","A2"],
    'Attribute B': ["B1"],
    'Attribute C': ["C1","C3"],
}

print (df1[list(filt)])
  Attribute A Attribute B Attribute C
0          A1          B1          C1
1          A2          B2          C2

mask = pd.concat([df1[k].isin(v) for k, v in filt.items()], axis=1).all(axis=1)
print (mask)
0     True
1    False
dtype: bool

df_filtered = df1[mask]
print (df_filtered)
  Attribute A Attribute B Attribute C  Price
0          A1          B1          C1    140