How to use pandas to_csv float_format?

Question

I am reading from a data file that has 8 precision, then after interpolating some values I am saving them like where the float_format option is not working,

df.to_csv('data.dat',sep=' ', index=False, header=False, float_format="%.8f")

and the result file looks like

0.02506602 0.05754493 0.36854688
0.02461631 0.0599653 0.43078098
0.02502534 0.06209149 0.44955311
0.4267356675182389 0.1718682822340447 0.5391386354945895
0.426701667727433 0.17191008887193007 0.5391897818631616
0.4266676661681287 0.17195189807522643 0.5392409104354972

The first 3 lines were in data file and next 3 are the new interpolated values. I want all the values to be of same length. Whats going wrong here and how do I fix it?

Also: It would be nice if I can control the float precision differently for different columns.

Answer 1

Your code looks fine. Most likely, there is an issue with your input data. Use pd.DataFrame.dtypes to check all your input series are of type float. If they aren't convert to float via:

df[col_list] = df[col_list].apply(pd.to_numeric, downcast='float').fillna(0)

Here's a working example:

from io import StringIO
import pandas as pd

mystr = StringIO("""0.02506602 0.05754493 0.36854688
0.02461631 0.0599653 0.43078098
0.02502534 0.06209149 0.44955311
0.4267356675182389 0.1718682822340447 0.5391386354945895
0.426701667727433 0.17191008887193007 0.5391897818631616
0.4266676661681287 0.17195189807522643 0.5392409104354972""")

df = pd.read_csv(mystr, delim_whitespace=True, header=None)

print(df.dtypes)

# 0    float64
# 1    float64
# 2    float64
# dtype: object

file_loc = r'C:\temp\test.dat'
df.to_csv(file_loc, sep=' ', index=False, header=False, float_format="%.8f")

df = pd.read_csv(file_loc, delim_whitespace=True, header=None)

print(df[0].iloc[-1])

# 0.42666767