Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to create a increment var from a first value of a dataframe group?

I have a datframe as :

data=[[0,1,5],
      [0,1,6],
      [0,0,8],
      [0,0,10],
      [0,1,12],
      [0,0,14],
      [0,1,16],
      [0,1,18],    
      [1,0,2],
      [1,1,0],
      [1,0,1],
      [1,0,2]]

df = pd.DataFrame(data,columns=['KEY','COND','VAL'])

For RES1, I want to create a counter variable RES where COND ==1. The value of RES for the first KEY of the group remains same as the VAL (Can I use cumcount() in some way).

For RES2, then I just want to fill the missing values as the previous value. (df.fillna(method='ffill')), I am thinking..

    KEY  COND    VAL RES1  RES2
0     0     1    5      5     5
1     0     1    6      6     6
2     0     0    8            6
3     0     0   10            6
4     0     1   12      7     7
5     0     0   14            7
6     0     1   16      8     8
7     0     1   18      9     9
8     1     0    2      2     2
9     1     1    0      3     3
10    1     0    1            3
11    1     0    2            3

Aim is to look fir a vectorized solution that's most optimal over million rows.

like image 862
flying_fluid_four Avatar asked Oct 23 '19 19:10

flying_fluid_four


Video Answer


1 Answers

IIUC

con=(df.COND==1)|(df.index.isin(df.drop_duplicates('KEY').index))
df['res1']=df.groupby('KEY').VAL.transform('first')+
           df.groupby('KEY').COND.cumsum()[con]-
           df.groupby('KEY').COND.transform('first')
df['res2']=df.res1.ffill()
df
   Out[148]: 
    KEY  COND  VAL  res1  res2
0     0     1    5   5.0   5.0
1     0     1    6   6.0   6.0
2     0     0    8   NaN   6.0
3     0     0   10   NaN   6.0
4     0     1   12   7.0   7.0
5     0     0   14   NaN   7.0
6     0     1   16   8.0   8.0
7     0     1   18   9.0   9.0
8     1     0    2   2.0   2.0
9     1     1    0   3.0   3.0
10    1     0    1   NaN   3.0
11    1     0    2   NaN   3.0
like image 142
BENY Avatar answered Sep 20 '22 01:09

BENY