Pandas - Comparing None values

Question

Consider this snippet:

a = pd.DataFrame([[None]])
b = pd.DataFrame([[None]])

Now, I want to validate both of them contains the exact same values:

int((a == b).sum())  # should be 1

but it's not 1. Instead, it returns 0. This behavior is giving me troubles especially in assert_frame_equal where it is reporting None is not None even though they are:

a.iloc[0,0] == b.iloc[0,0]  # True

Why is that and how can I fix this?

Answer 1

pandas is special casing None so as to be interpreted as NaN (since NaN != NaN, and pd.isnull treats both consistently... this may be one explanation).

Not a solution, but a workaround – np.array_equal works, if they're None and not NaN;

>>> np.array_equal(a, b)
True

If you want the count and not a bool result, use np.equal;

>>> np.equal(a, b).sum().item()
1