Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas.DataFrame.rolling not working with huge floats

I have an error with pandas rolling when using floats close to infinity. I display an example here:

import pandas as pd
series = pd.Series(1.,index = pd.date_range('2015-01-01', periods=6))
series[series.index[2]] = 1e19
series
2015-01-01    1.000000e+00
2015-01-02    1.000000e+00
2015-01-03    1.000000e+19
2015-01-04    1.000000e+00
2015-01-05    1.000000e+00
2015-01-06    1.000000e+00
Freq: D, dtype: float64
series.rolling('2D', closed = 'left').mean()
2015-01-01             NaN
2015-01-02    1.000000e+00
2015-01-03    1.000000e+00
2015-01-04    5.000000e+18
2015-01-05    5.000000e+18
2015-01-06    5.000000e-01
Freq: D, dtype: float64

The answer in the last bit should be 1! but it is 0.5. Why does rolling go nuts when using big numbers? same example with smaller float:

series[series.index[2]] = 1e9
series.rolling('2D', closed = 'left').mean()
2015-01-01            NaN
2015-01-02            1.0
2015-01-03            1.0
2015-01-04    500000000.5
2015-01-05    500000000.5
2015-01-06            1.0
Freq: D, dtype: float64
like image 332
karen Avatar asked Oct 11 '17 13:10

karen


1 Answers

The problem is not with pandas. I tried the same thing in R with rollmean function and it gives the exact same result as pandas. It does not work for value of 1e16 and above. I thing it has to do with how the system handles float.

like image 186
Pal Avatar answered Oct 10 '22 17:10

Pal