Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Updating dataframe ID w.r.t missing dates column values

Tags:

python

pandas

I am trying to update the ID of dataframe with respect to the missing days of date column in dataframe,

        Date    ID
0   2018-01-01  45.0-A
1   2018-01-02  45.0-A
5   2018-01-06  45.0-A
6   2018-01-07  45.0-A
12  2018-01-13  45.0-A
13  2018-01-14  45.0-A

period = 2

If the dataframe has more than specified period (period =2 )of days missing ID should updated with extra number, I solved this with time difference and looping over dataframe, it is taking more time. Can someone suggest me the most efficient way to achieve this?

T_diff = data.Date.diff()
slic = [data.index[0]] + T_diff[T_diff.dt.days>period].index.tolist() + [data.index[-1]]
li = []
for i in range(len(slic)-1):
    temp_df = data.loc[slic[i]:slic[i+1]].copy()
    temp_df['ID'] = temp_df['ID'] + '_{}'.format(i)
    li.append(temp_df)
pd.concat(li,axis=0)

         Date   ID
0   2018-01-01  45.0-A_0
1   2018-01-02  45.0-A_0
5   2018-01-06  45.0-A_1
6   2018-01-07  45.0-A_1
12  2018-01-13  45.0-A_2
13  2018-01-14  45.0-A_2
like image 483
Naga kiran Avatar asked Aug 17 '26 16:08

Naga kiran


1 Answers

This can be done in one line, using diff() and cumsum()

df['Date'] = pd.to_datetime(df['Date'])
df['ID'] += '_' + (df['Date'].diff() > pd.Timedelta('2D')).cumsum().astype(str)


#output
#         Date        ID
#0  2018-01-01  45.0-A_0
#1  2018-01-02  45.0-A_0
#5  2018-01-06  45.0-A_1
#6  2018-01-07  45.0-A_1
#12 2018-01-13  45.0-A_2
#13 2018-01-14  45.0-A_2

like image 57
Jondiedoop Avatar answered Aug 20 '26 05:08

Jondiedoop



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!