Pandas unit testing: How to assert equality of NaT and NaN values?

Question

In NumPy and Pandas, nan != nan and NaT != NaT. So, when comparing results during unit testing, how can I assert that a returned value is one of those values? A simple assertEqual naturally fails, even if I use pandas.util.testing.

Answer 1

If you're comparing scalars, one way is to use assertTrue with isnull. For example, in the DataFrame unit tests (pandas/tests/test_frame.py) you can find tests such as this:

self.assertTrue(com.isnull(df.ix['c', 'timestamp']))

(com is an alias for pandas/core/common.py and so com.isnull calls the same underlying function as pd.isnull.)

If on the other hand you're comparing Series or DataFrames with null values for equality, these are handled automatically by tm.assert_series_equal and tm.assert_frame_equal. For example:

>>> import pandas.util.testing as tm
>>> df = pd.DataFrame({'a': [1, np.nan]})
>>> df
    a
0   1
1 NaN

Normally, NaN is not equal to NaN:

>>> df == df
       a
0   True
1  False

But assert_frame_equal processes NaN as being equal to itself:

>>> tm.assert_frame_equal(df, df)
# no AssertionError raised