Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Filtering pandas dataframe rows by contains str

I have a python pandas dataframe df with a lot of rows. From those rows, I want to slice out and only use the rows that contain the word 'ball' in the 'body' column. To do that, I can do:

df[df['body'].str.contains('ball')]

The issue is, I want it to be case insensitive, meaning that if the word Ball or bAll showed up, I'll want those as well. One way to do case insensitive search is to turn the string to lowercase and then search that way. I'm wondering how to go about doing that. I tried

df[df['body'].str.lower().contains('ball')]

But that doesn't work. I'm not sure if I'm supposed to use a lambda function on this or something of that nature.

like image 512
David Avatar asked Sep 16 '15 18:09

David


People also ask

How do you check if a row contains a string in pandas?

Using “contains” to Find a Substring in a Pandas DataFrame The contains method returns boolean values for the Series with True for if the original Series value contains the substring and False if not. A basic application of contains should look like Series. str. contains("substring") .

How do I filter specific rows from a DataFrame pandas?

Filter Rows by Condition You can use df[df["Courses"] == 'Spark'] to filter rows by a condition in pandas DataFrame. Not that this expression returns a new DataFrame with selected rows.


1 Answers

You could either use .str again to get access to the string methods, or (better, IMHO) use case=False to guarantee case insensitivity:

>>> df = pd.DataFrame({"body": ["ball", "red BALL", "round sphere"]})
>>> df[df["body"].str.contains("ball")]
   body
0  ball
>>> df[df["body"].str.lower().str.contains("ball")]
       body
0      ball
1  red BALL
>>> df[df["body"].str.contains("ball", case=False)]
       body
0      ball
1  red BALL
>>> df[df["body"].str.contains("ball", case=True)]
   body
0  ball

(Note that if you're going to be doing assignments, it's a better habit to use df.loc, to avoid the dreaded SettingWithCopyWarning, but if we're just selecting here it doesn't matter.)

(Note #2: guess I really didn't need to specify 'round' there..)

like image 70
DSM Avatar answered Oct 24 '22 05:10

DSM