I have the following data in a dataframe.
| Timestamp | MeasureA | MeasureB | MeasureC | MeasureD |
|---|---|---|---|---|
| 0.00 | 26.46 | 63.60 | 3.90 | 0.67 |
| 0.94 | 26.52 | 78.87 | 1.58 | 0.42 |
| 1.94 | 30.01 | 82.04 | 1.13 | 0.46 |
| 3.00 | 30.19 | 82.00 | 1.17 | 0.36 |
| 4.00 | 30.07 | 81.43 | 1.13 | 0.42 |
| 5.94 | 30.02 | 82.46 | 1.05 | 0.34 |
| 8.00 | 30.22 | 82.48 | 0.98 | 0.35 |
| 9.00 | 30.00 | 82.21 | 1.13 | 0.33 |
| 10.00 | 30.00 | 82.34 | 1.12 | 0.34 |
And I'd like to filter the entries using some non-uniform intervals. Let say that my intervals are [1.0, 1.5]
What I'm trying to achieve is that, we take the first row (row0), and to get the next valid row, we look at what is the next row whose Timestamp value is greater or equal to row0 + 1.0.
In this scenario, the next valid row will be the one with the 1.94 timestamp. Then, for the next valid row, we will use the next item in the intervals array. Which is 1.5. That will make the next row the one with a timestamp value of 4.00. Since 1.94 + 1.5 is equal to 3.44.
For the next row, we go back and start from the beginning of the array of intervals.
After going through all the data, the resulting dataframe should be:
| Timestamp | MeasureA | MeasureB | MeasureC | MeasureD |
|---|---|---|---|---|
| 0.00 | 26.46 | 63.60 | 3.90 | 0.67 |
| 1.94 | 30.01 | 82.04 | 1.13 | 0.46 |
| 4.00 | 30.07 | 81.43 | 1.13 | 0.42 |
| 5.94 | 30.02 | 82.33 | 1.11 | 0.35 |
| 8.00 | 30.22 | 82.48 | 0.98 | 0.35 |
| 9.00 | 30.00 | 82.21 | 1.13 | 0.33 |
Is there a way to achieve this with the existing filtering methods in pandas?
Try:
from itertools import cycle
# the interval:
A, B = 1.0, 1.5
comparing, out, last_t = cycle([B, A]), [], float("-inf")
j = next(comparing)
for i, t in zip(df.index, df.Timestamp):
if t >= last_t + j:
out.append(i)
last_t = t
j = next(comparing)
print(df.loc[out])
Prints:
Timestamp MeasureA MeasureB MeasureC MeasureD
0 0.00 26.46 63.60 3.90 0.67
2 1.94 30.01 82.04 1.13 0.46
4 4.00 30.07 81.43 1.13 0.42
5 5.94 30.02 82.46 1.05 0.34
6 8.00 30.22 82.48 0.98 0.35
7 9.00 30.00 82.21 1.13 0.33
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With