Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas dataframe filter to return True for ALL rows. how?

Hi I have a filter 'm' set that is flexible enough to change by me. Sometimes, I want to filter by Car or x_acft_body , or any of the various other fields, etc. Sometime I want to have all of the rows returned by commenting and uncommenting the required lines. But without changing the subsequent code, after the filter 'm' line.

How can I have a filter that will return true for ALL rows, when I don't want the filters applied? For e.g. something like 1==1 but i know this doesn't work.

I don't want to set dfdata.somefield.notnull() etc. as I will not be too sure if this field will be always not null or not. also I DO NOT want to change subsequent code to be like dfdata.groupby. i.e. without the [m]

# set filter if needed
m = (   1==1 #& return true at all times
#         (dfdata.Car == 'PG') #&
#         (dfdata.x_acft_body == 'N')# &
#         (dfdata.Car.isin(['PG', 'VJ', 'VZ']))
)


dft1 = dfdata[m].groupby(['FLD1']).agg({'FLD2': 'count'})
like image 332
ihightower Avatar asked Sep 24 '17 07:09

ihightower


People also ask

How do I make pandas show all rows?

Set value of display. max_rows to None and pass it to set_option and this will display all rows from the data frame.

How do you filter DataFrame rows based on condition?

Filter Rows by Condition You can use df[df["Courses"] == 'Spark'] to filter rows by a condition in pandas DataFrame. Not that this expression returns a new DataFrame with selected rows. You can also write the above statement with a variable.

Which of the following will filter rows in a pandas DataFrame multiple correct?

We will select multiple rows in pandas using multiple conditions, logical operators and using loc() function. Selecting rows with logical operators i.e. AND and OR can be achieved easily with a combination of >, <, <=, >= and == to extract rows with multiple filters.


1 Answers

You can create bool constant and change final mask by it:

#True for return all rows
m = (dfdata.Car == 'PG') | True

And:

#False for apply filter
m = (dfdata.Car == 'PG') | False

First solutions:

m = [True] * len(df.index)

m = np.repeat(True, len(df.index))
like image 142
jezrael Avatar answered Oct 04 '22 02:10

jezrael