Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to get the percent change of values in a dataframe while caring about NaN values?

I have the following DataFrame:

   Date                 A                
   2015-01-01          10               
   2015-01-02          14               
   2015-01-05          NaN               
   2015-01-06          NaN                
   2015-01-07          5
   2015-01-10          1  
   2015-01-11          NaN  
   2015-01-12          21  
   2015-01-14          13 

And I would like to get a data frame with the pct change in the data frame only if this two values are adjacent and not separated by NaN.

i.e. I would like this:

        Date                 A                             
       2015-01-02         0.2857                           
       2015-01-10         -0.8  
       2015-01-14         -0.38

But if I do: df.pct_change() it will return the pct change between 14 and 5 for the 201-01-07

like image 494
astudentofmaths Avatar asked Apr 20 '17 18:04

astudentofmaths


2 Answers

Use pct_change and shift:

df.pct_change()[df.shift(1).notnull()].dropna()

Output:

                   A
Date                
2015-01-02  0.400000
2015-01-10 -0.800000
2015-01-14 -0.380952
like image 56
Scott Boston Avatar answered Oct 18 '22 22:10

Scott Boston


use cumsum on isnull to find groups to groupby

s = df.set_index('Date').A
c = n.cumsum()
s.groupby(c).pct_change().dropna().reset_index()

        Date         A
0 2015-01-02  0.400000
1 2015-01-10 -0.800000
2 2015-01-14 -0.380952
like image 37
piRSquared Avatar answered Oct 18 '22 22:10

piRSquared