python pandas.Series.isin with case insensitive

Question

I want to filter out some rows with one of DataFrame's column which data is in a list.

df[df['column'].isin(mylist)]

But I found that it's case sensitive. Is there any method using ".isin()" with case insensitive?

Answer 1

One way would be by comparing the lower or upper case of the Series with the same for the list

df[df['column'].str.lower().isin([x.lower() for x in mylist])]

The advantage here is that we are not saving any changes to the original df or the list making the operation more efficient

Consider this dummy df:

    Color   Val
0   Green   1
1   Green   1
2   Red     2
3   Red     2
4   Blue    3
5   Blue    3

For the list l:

l = ['green', 'BLUE']

You can use isin()

df[df['Color'].str.lower().isin([x.lower() for x in l])]

You get

    Color   Val
0   Green   1
1   Green   1
4   Blue    3
5   Blue    3

Answer 2

I prefer to use the general .apply

myset = set([s.lower() for s in mylist])
df[df['column'].apply(lambda v: v.lower() in myset)]

A lookup in a set is faster than a lookup in a list