pandas query None values [duplicate]

Question

I have a dataframe with columns of different dtypes and I need to use pandas.query to filter the columns.

Columns can include missing values: NaN, None and NaT and I need to display the rows that contain such values. Is there a way to do this in an expression passed to pandas.query? I am aware that it can be done using different methods but I need to know if it is doable through the query

For boolean columns I was able to use a workaround by stating:

df.query('col not in (True, False)')

but this won't work for other types of columns. Any help is appreciated including workarounds.

Answer 1

NaN is not equal to itself, so you can simply test if a column is equal to itself to filter it. This also seems to work for None although I'm not sure why, it may be getting cast to NaN at some point during the evaluation.

 df.query('col == col')

For datetimes, this works, but feels pretty hacky, there might be a better way.

df.query('col not in [@pd.NaT]')