Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How do I filter out multiple columns witha certain string in Python

Tags:

python

pandas

I'm new to python and especially to pandas so I don't really know what I'm doing. I have 10 columns with 100000 rows and 4 letter strings. I need to filter out rows which don't contain 'DDD' in all of the columns/rows.

I tried to do it with iloc and loc, but it doesn't work:

import pandas as pd
df = pd.read_csv("data_3.csv", delimiter = '!')
df.iloc[:,10:20].str.contains('DDD', regex= False, na = False)
df.head()

It returns me an error: 'DataFrame' object has no attribute 'str'

like image 833
Goldust34 Avatar asked Aug 15 '26 03:08

Goldust34


2 Answers

I suggest doing it without a for loop like this:

df[df.apply(lambda x: x.str.contains('DDD')).all(axis=1)]

To select only string columns

df[df.select_dtypes(include='object').apply(lambda x: x.str.contains('DDD')).all(axis=1)]

To select only some string columns

selected_cols = ['A','B']
df[df[selected_cols].apply(lambda x: x.str.contains('DDD')).all(axis=1)]
like image 149
Christian Sloper Avatar answered Aug 16 '26 16:08

Christian Sloper


You can do this but if your all column type is StringType:

for column in foo.columns:
    df = df[~df[c].str.contains('DDD')]
like image 34
yasi Avatar answered Aug 16 '26 18:08

yasi