Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pandas df.corr() returns NaN despite data fed having populated data

Tags:

python

pandas

My data looks like the following:

print data

                   A                B 
2014-04-04        163.24         191.77
2014-06-11        165.43         182.25
2014-12-22        194.44         161.44
2014-03-25        163.25         195.04
2014-11-03        190.83         164.36
2014-06-16        165.85         182.35
2014-11-24        190.07         162.15
2014-05-21        159.35         186.39
2015-01-21        177.15         152.09
2014-08-28        177.65         192.00
2014-02-19        163.26         182.95
2014-05-12        159.55         192.57
2014-07-09        164.67         188.42
2015-02-26        192.20         160.87
2014-10-14        178.70         183.80
2014-07-16        170.47         192.36
2014-01-21        173.20         188.43
2014-06-03        161.80         184.37
2014-03-17        166.84         185.81
2014-08-11        172.46         187.47
2015-01-07        187.28         155.05
2014-05-29        160.74         183.76
2015-02-11        187.65         158.20
2014-02-27        165.38         185.27
2015-01-05        188.34         159.51
2014-07-01        166.81         186.35
2015-02-18        188.67         162.19
2014-04-03        166.09         192.69
2014-10-06        187.48         189.04
2014-12-29        196.19         160.51

When I try to call data.corr() I receive back all NaN. I tested this in excel, and the correlation is indeed able to be calculated.

Any idea why this might be the case?

I am happy to post more but i am unsure what would help.

print data.info()

<class 'pandas.core.frame.DataFrame'>
Index: 298 entries, 2014-04-04 to 2014-01-29
Data columns (total 2 columns):
A     298 non-null float64
B    298 non-null float64
dtypes: float64(2)None



print data.describe()
              A                B
    count    298.000000     298.000000
    mean     175.152886     178.596242
    std       11.490668      13.709316
    min      152.720000     151.410000
    25%      165.502500     162.655000
    50%      175.480000     184.330000
    75%      185.085000     189.692500
    max      196.890000     197.770000


print pd.version
0.14.1
like image 355
jason m Avatar asked Mar 13 '15 21:03

jason m


1 Answers

Jason, Using the sample data it works fine for me.

data.corr()
    A   B
A   1.000000    -0.779864
B   -0.779864   1.000000

I noticed in your data.info() there is a 'None' tacked on to dtypes. When I load your sample data I get

data.info()
<class 'pandas.core.frame.DataFrame'>
Index: 30 entries, 2014-04-04 to 2014-12-29
Data columns (total 2 columns):
A     30 non-null float64
B     30 non-null float64
dtypes: float64(2)

I imagine what's causing that 'None' is causing your problem.

Hope this helps [wish I could leave as a comment but don't have the reputation yet!]

like image 156
EngineeredE Avatar answered Nov 14 '22 21:11

EngineeredE