Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cleaner pandas/numpy code to find equivalency matrix?

I have pandas DataFrame and would like to generate an equivalency matrix (or whatever it's called) where each cell has one value if the the df.Col[i] == df.Col[j] and another value when !=.

The following code works:

df = pd.DataFrame({"Col":[1, 2, 3, 1, 2]}, index=["A","B","C","D","E"])
df

    Col
A   1
B   2
C   3
D   1
E   2

sm = pd.DataFrame(columns=df.index, index=df.index)
for i in df.index:
    for j in df.index:
        if df.Col[i] == df.Col[j]:
            sm.loc[i, j] = 3
        else:
            sm.loc[i, j] = -1
sm

     A   B   C   D   E
A    3  -1  -1   3  -1
B   -1   3  -1  -1   3
C   -1  -1   3  -1  -1
D    3  -1  -1   3  -1
E   -1   3  -1  -1   3

But there must be a better way. Perhaps using numpy? Any thoughts?

[Edit]

Using what piRsquared wrote, perhaps something like?

m = df.values == df.values[:, 0]
sm = pd.DataFrame(None, df.index, df.index).where(m, 3).where(~m, -1)

Can this be improved?

like image 674
Robert Mah Avatar asked May 07 '26 22:05

Robert Mah


2 Answers

v = df.values
m = v == v[:, 0]
pd.DataFrame(np.where(m, 1, -1), df.index, df.index)

   A  B  C  D  E
A  1 -1 -1  1 -1
B -1  1 -1 -1  1
C -1 -1  1 -1 -1
D  1 -1 -1  1 -1
E -1  1 -1 -1  1
like image 198
piRSquared Avatar answered May 10 '26 12:05

piRSquared


#initialize your sm to 1s
sm = pd.DataFrame(columns=df.index, index=df.index, data=1)
#create a mask to indicate equivalence
mask = (np.asarray(df)[:,None]==np.asarray(df)).reshape(5,5)
#set non-equivalent elements to -1
sm = sm.where(mask,-1)
sm
Out[129]: 
   A  B  C  D  E
A  1 -1 -1  1 -1
B -1  1 -1 -1  1
C -1 -1  1 -1 -1
D  1 -1 -1  1 -1
E -1  1 -1 -1  1
like image 20
Allen Avatar answered May 10 '26 11:05

Allen



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!