How can I stop Python's csv.DictWriter.writerows from adding empty lines between rows in Windows when using stdout?

Question

When I run this code in Eclipse using PyDev, I see output with empty lines in between the rows. How can I get rid of them? It happens using python3 only.

import csv
import sys

fieldnames = ['first_name', 'last_name']
writer = csv.DictWriter(sys.stdout, fieldnames=fieldnames)

writer.writeheader()
writer.writerow({'first_name': 'Baked', 'last_name': 'Beans'})
writer.writerow({'first_name': 'Lovely', 'last_name': 'Spam'})
writer.writerow({'first_name': 'Wonderful', 'last_name': 'Spam'})

Output in Eclipse

first_name,last_name

Baked,Beans

Lovely,Spam

Wonderful,Spam

Output is correct in cygwin-bash, on Linux and incorrect when using redirection to file in cmd.exe + Windows python (Console window is apparently correct)

Answer 1

The documentation is pretty clear that the CSV module needs to have control over the newlines; in Python 2 you are expected to open the file object in binary mode, in Python 3, you need to set the newline option to ''. See How can I stop Python's csv.DictWriter.writerows from adding empty lines between rows in Windows?

That's because the module writes \r\n line separators to the file by default. The windows text mode handling then replaces \n with \r\n and you see blank lines appear because now lines are separated by \r\r\n.

You can set the lineterminator argument to '\n' to change what the module uses to separate lines:

writer = csv.DictWriter(sys.stdout, fieldnames=fieldnames, 
                        lineterminator='\n')

Demo:

>>> import csv
>>> from StringIO import StringIO
>>> fieldnames = ['first_name', 'last_name']
>>> out = StringIO()
>>> writer = csv.DictWriter(out, fieldnames=fieldnames)
>>> writer.writeheader()
>>> out.getvalue()
'first_name,last_name\r\n'
>>> out = StringIO()
>>> writer = csv.DictWriter(out, fieldnames=fieldnames, lineterminator='\n')
>>> writer.writeheader()
>>> out.getvalue()
'first_name,last_name\n'

sys.stdout will still translate that \n to \r\n but now there is no \r preceding it.