Pandas check if value in one multiindex column is in any column, same row of different multiindex

Question

I have the following dataframe:

df = pd.DataFrame([[0, 1, 7, 0, 1, 8, 3, 0],
                   [7, 3, 4, 0, 4, 9, 7, 0]], 
                  columns=pd.MultiIndex.from_product([["first", "second"], 
                                                      ["A", "B", "C", "D"]]))
print(df)

  first          second         
      A  B  C  D      A  B  C  D
0     0  1  7  0      1  8  3  0
1     7  3  4  0      4  9  7  0

I want to check, whether the values in first are present in any of the columns of second. Only the same row should be compared.

The resulting dataframe should look like this:

      A      B      C     D
0  True   True  False  True
1  True  False   True  True

What is the best way of doing this? I have already played around with df["first"].isin(df["second"] but it only compares A with A, B with B, ... Also tried it in combination with .any() but I can't seem to make it work.

Your help is greatly appreciated!

Thank you in advance.

Answer 1

Numpy broadcasting

np.any(df['first'].T.values[:, :, None] == df['second'].values, axis=-1).T

---

array([[ True,  True, False,  True],
       [ True, False,  True,  True]])

Answer 2

Another solution:

df['first'].apply(lambda x: x.isin(df.loc[x.name, ('second')]), axis=1)

Output:

    A         B       C      D
0   True    True    False   True
1   True    False   True    True