Prevent pandas from reading None as Nan

Question

I have cleaned a dataset and had to replace a lot of NaN values with None. After that I saved it to a new csv file, when I read the cleaned dataset back using pandas.read_csv, all the None values are represented as NaN, how can I avoid this?

Answer 1

You can use parameter keep_default_na and na_values in read_csv and then replace strings None to values None:

import pandas as pd
from pandas.compat import StringIO

temp=u"""a,b
None,NaN
a,8"""
#after testing replace 'StringIO(temp)' to 'filename.csv'
df = pd.read_csv(StringIO(temp),keep_default_na=False,na_values=['NaN'])

print (df)
      a    b
0  None  NaN
1     a  8.0

print (type(df.a.iloc[0]))
<class 'str'>

df = df.replace({'None':None})
print (df)
      a    b
0  None  NaN
1     a  8.0

print (type(df.a.iloc[0]))
<class 'NoneType'>