Pandas DataFrame Replace NaT with None

Question

I have been struggling with this question for a long while, and I tried different methods.

I have a simple DataFrame as shown,

I can use code to replace NaN with None (Not String "None"),

[![dfTest2 = dfTest.where(pd.notnull(dfTest), None)][2]][2]

I support that NaT is also classified as 'Null' because the following,

However, NaT is not replaced with None.

I have been searching for answers but got no luck. Anyone could Help?

Thank you in advance.

Answer 1

Make the dtype object

dfTest2 = pd.DataFrame(dict(InvoiceDate=pd.to_datetime(['2017-06-01', pd.NaT])))  dfTest2.InvoiceDate.astype(object).where(dfTest2.InvoiceDate.notnull(), None)  0    2017-06-01 00:00:00 1                   None Name: InvoiceDate, dtype: object 

Answer 2

The simplest solution I found that worked for me is...

Input:

import pandas as pd import numpy as np dfTest = pd.DataFrame(dict(InvoiceDate=pd.to_datetime(['2017-06-01', pd.NaT]), CorpId=[2997373, np.nan], TestName=[1,1])) dfTest.replace({np.nan: None}, inplace = True) 

Output of dfTest:

Answer 3

Make the column type as str first

 dfTest2.InvoiceDate =  dfTest2.InvoiceDate.astype(str)

then compare it directly with "NaT" and replace with None

dfTest2.InvoiceDate = dfTest2.InvoiceDate.apply(lambda x : None if x=="NaT" else x)

Answer 4

Similar approach as suggested by @neerajYadav but without the apply:

dfTest2['InvoiceDate'] = (dfTest2['InvoiceDate']
                          .astype(str) # <- cast to string to simplify
                                       #    .replace() in newer versions
                          .replace({'NaT': None} # <- replace with None
                         )