How do I check for correlation using Decimal numbers/data with python 3

Question

Thank you for your time.

I am writing some code that is checking for correlation between multiple sets of data. It works great when I am using the original data (which I am honestly unsure of which format it is in at that point), but after I run the data through some equations using the Decimal module, the data set will not show up when tested for correlation.

I feel really stupid and new lol, I am sure it's a very easy fix.

Here is a small program I wrote to demonstrate what I mean.

from decimal import Decimal
import numpy as np
import pandas as pd

a = [Decimal(2.3), Decimal(1.5), Decimal(5.7), Decimal(4.6), Decimal(5.5), Decimal(1.5)]
b = [Decimal(2.1), Decimal(1.2), Decimal(5.3), Decimal(4.4), Decimal(5.3), Decimal(1.7)]

h = [2.3,1.5,5.7,4.6,5.5,1.5]
j = [2.1,1.2,5.3,4.4,5.3,1.7]

corr_data1 = pd.DataFrame({'A': a, 'B': b}) 

corr_data2 = corr_data1.corr()
print(corr_data2)

corr_data3 = pd.DataFrame({'H': h, 'J': j})

corr_data4 = corr_data3.corr()
print(corr_data4)

The data for both lists A & B as well as H & F are exactly the same, with the only difference of A & B being decimal formated numbers, where as H & F are not.

When the program is run, A & B returns:

Empty DataFrame
Columns: []
Index: []

and H & J returns:

          H         J
H  1.000000  0.995657
J  0.995657  1.000000

How do I make it so I can utilize the data after I've ran it through my equations?

Sorry for the stupid question and thank you for your time. I hope you are all well, happy holidays!

Answer 1

Pandas does not recognize the data as numeric values. Here is how to convert your data to float.

corr_data1.astype(float).corr()

#           A         B
# A  1.000000  0.995657
# B  0.995657  1.000000

This should also work but it actually does not.

pd.to_numeric(corr_data1['A'], errors='coerce')

# 0   NaN
# 1   NaN
# 2   NaN
# 3   NaN
# 4   NaN
# 5   NaN