In a time series (ordered tuples), what's the most efficient way to find the first time a criterion is met?
In particular, what's the most efficient way to determine when a value goes over 100 for the value of a column in a pandas data frame?
I was hoping for a clever vectorized solution, and not having to use df.iterrows()
.
For example, for price or count data, when a value exceeds 100. I.e. df['col'] > 100.
price
date
2005-01-01 98
2005-01-02 99
2005-01-03 100
2005-01-04 99
2005-01-05 98
2005-01-06 100
2005-01-07 100
2005-01-08 98
but for potentially very large series. Is it better to iterate (slow) or is there a vectorized solution?
A df.iterrows()
solution could be:
for row, ind in df.iterrows():
if row['col'] > value_to_check:
breakpoint = row['value_to_record'].loc[ind]
return breakpoint
return None
But my question is more about efficiency (potentially, a vectorized solution that will scale well).
By using the pandas series.gt() method we can check if the elements of a series object are Greater Than a scalar value or not. The gt() comparison operation is exactly equivalent to series > Other.
le (equivalent to <= ) — less than or equals to. lt (equivalent to < ) — less than. ge (equivalent to >= ) — greater than or equals to. gt (equivalent to > ) — greater than.
Pandas DataFrame max() Method The max() method returns a Series with the maximum value of each column. By specifying the column axis ( axis='columns' ), the max() method searches column-wise and returns the maximum value for each row.
to get the top 5 most occuring values use df['column']. value_counts(). head(n) and the solution provided by @lux7 df['column']. nlargest(n=5) would result in the top 5 values from a column(their values not how many times they have appeared).
Try this: "> 99"
df[df['price'].gt(99)].index[0]
returns "2"
, the second index row.
all row indexes greater than 99
df[df['price'].gt(99)].index
Int64Index([2, 5, 6], dtype='int64')
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With