Remove NaT values while inserting data into mongo using pymongo

Question

Have a df with values

name    date

tom    2019-12-10 07:44:03.733
mark   NaT

if i try this :

data = df.to_dict("records")

mydb.test.insert_many(data)

I'm getting this error :

ValueError: NaTType does not support utcoffset

How to send this data to mongodb without NaT and keeping the column empty without converting to string.

expected data in mongodb :

{
_id:ObjectId(5db012b123a2a1cabcc12345),
name:tom,
date:2019-10-23T10:55:50.569+00:00
}


{
_id:ObjectId(5db012b123a2a1cabcc12346),
name:mark
}

Answer 1

You can convert NaT to None, like below:

If your df['date'] is not a datetime column, convert it into one.

df['date'] = pd.to_datetime(df['date'])

df['date'] = df['date'].astype(object).where(df['date'].notnull(), None)

Then you can insert this in mongo.