I want to ask for ideas of an elegant(time saving) way to reduce redundant digitally sampled data. I got a .csv files with a structure like that:
time, data1, data2, data3
0.1, 1, 1, 0
0.2, 1, 1, 0
0.3, 1, 1, 0
0.4, 1, 0, 0
0.5, 1, 0, 0
0.6, 1, 0, 0
0.7, 0, 1, 0
0.8, 0, 1, 0
I want to reduce the incoming data in such manner that all unnecessary time steps are removed. E.g.
time, data1, data2, data3
0.1, 1, 1, 0
0.4, 1, 0, 0
0.7, 0, 1, 0
The remaining data should just consist of the timesteps whenever the bitcode has changed. I already implemented a method importing the csv file with pandas and using the corresponding mask and compress functions. But the way I implemented it is very time consuming.
If first column is index use:
df = df[df.ne(df.shift()).any(axis=1)]
print (df)
data1 data2 data3
time
0.1 1 1 0
0.4 1 0 0
0.7 0 1 0
If first column is not index:
df1 = df.iloc[:, 1:]
df = df[df1.ne(df1.shift()).any(axis=1)]
print (df)
time data1 data2 data3
0 0.1 1 1 0
3 0.4 1 0 0
6 0.7 0 1 0
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With