Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Flag only first row where condition is met in a DataFrame

I have the following DataFrame df, which can be created as follows:

date_today = datetime.now().date()
days = pd.date_range(date_today, date_today + timedelta(19), freq='D')
x = np.arange(0,2*np.pi,0.1*np.pi)   # start,stop,step
y = np.sin(x)
df = pd.DataFrame({'dates': days, 'vals': y, 'is_hit': abs(y)>0.9})
df = df.set_index('dates')

And which looks like this:

            is_hit          vals
dates                           
2019-03-27   False  0.000000e+00
2019-03-28   False  3.090170e-01
2019-03-29   False  5.877853e-01
2019-03-30   False  8.090170e-01
2019-03-31    True  9.510565e-01
2019-04-01    True  1.000000e+00
2019-04-02    True  9.510565e-01
2019-04-03   False  8.090170e-01
2019-04-04   False  5.877853e-01
2019-04-05   False  3.090170e-01
2019-04-06   False  1.224647e-16
2019-04-07   False -3.090170e-01
2019-04-08   False -5.877853e-01
2019-04-09   False -8.090170e-01
2019-04-10    True -9.510565e-01
2019-04-11    True -1.000000e+00
2019-04-12    True -9.510565e-01
2019-04-13   False -8.090170e-01
2019-04-14   False -5.877853e-01
2019-04-15   False -3.090170e-01

I want to flag the rows where the is_hit condition is True for the first time, such that the expected new column hit_first would be:

           is_hit          vals  hit_first
dates                                      
2019-03-27   False  0.000000e+00      False
2019-03-28   False  3.090170e-01      False
2019-03-29   False  5.877853e-01      False
2019-03-30   False  8.090170e-01      False
2019-03-31    True  9.510565e-01       True
2019-04-01    True  1.000000e+00      False
2019-04-02    True  9.510565e-01      False
2019-04-03   False  8.090170e-01      False
2019-04-04   False  5.877853e-01      False
2019-04-05   False  3.090170e-01      False
2019-04-06   False  1.224647e-16      False
2019-04-07   False -3.090170e-01      False
2019-04-08   False -5.877853e-01      False
2019-04-09   False -8.090170e-01      False
2019-04-10    True -9.510565e-01       True
2019-04-11    True -1.000000e+00      False
2019-04-12    True -9.510565e-01      False
2019-04-13   False -8.090170e-01      False
2019-04-14   False -5.877853e-01      False
2019-04-15   False -3.090170e-01      False

How to create this hit_first column?

like image 460
JejeBelfort Avatar asked Mar 27 '19 12:03

JejeBelfort


2 Answers

My suggestion:

df['hit_first'] = df['is_hit'] & (~df['is_hit']).shift(1)
like image 132
ecortazar Avatar answered Oct 09 '22 09:10

ecortazar


Use Series.shift chained with & for bitwise AND:

df['hit_first'] = df['is_hit'].ne(df['is_hit'].shift()) & df['is_hit']
print (df)
                    vals  is_hit  hit_first
dates                                      
2019-03-27  0.000000e+00   False      False
2019-03-28  3.090170e-01   False      False
2019-03-29  5.877853e-01   False      False
2019-03-30  8.090170e-01   False      False
2019-03-31  9.510565e-01    True       True
2019-04-01  1.000000e+00    True      False
2019-04-02  9.510565e-01    True      False
2019-04-03  8.090170e-01   False      False
2019-04-04  5.877853e-01   False      False
2019-04-05  3.090170e-01   False      False
2019-04-06  1.224647e-16   False      False
2019-04-07 -3.090170e-01   False      False
2019-04-08 -5.877853e-01   False      False
2019-04-09 -8.090170e-01   False      False
2019-04-10 -9.510565e-01    True       True
2019-04-11 -1.000000e+00    True      False
2019-04-12 -9.510565e-01    True      False
2019-04-13 -8.090170e-01   False      False
2019-04-14 -5.877853e-01   False      False
2019-04-15 -3.090170e-01   False      False
like image 29
jezrael Avatar answered Oct 09 '22 11:10

jezrael