portable way to write csv file in python 2 or python 3

Question

On my Windows box, I usually did this in python 2 to write a csv file:

import csv
f = open("out.csv","wb")
cr = csv.writer(f,delimiter=';')
cr.writerow(["a","b","c"])
f.close()

Now that python 3 forbids writing text files as binary, that piece of code does not work anymore. That works:

import csv
f = open("out.csv","w",newline='')
cr = csv.writer(f,delimiter=';')
cr.writerow(["a","b","c"])
f.close()

Problem is: newline parameter is unknown to Python 2.

Of course, omitting the newline results in a csv file with too many \r chars, so not acceptable.

I'm currently performing a backwards compatible process to progressively migrate from python 2 to python 3.5 There are a lot of those statements in all my modules.

My solution was embedding the code in a custom module, and the custom module returns file handler + writer object. A python version check is done inside the module, which allows any module using my module to work whatever python version without too much hacking.

Is there a better way?

Answer 1

On Windows, I found a python 2 & 3 compliant way of doing it changing csv lineterminator option (which defaults to "\r\n" which makes one \r too many when file is open in text mode in Windows)

import csv

with open("out.csv","w") as f:
    cr = csv.writer(f,delimiter=";",lineterminator="\n")
    cr.writerow(["a","b","c"])
    cr.writerow(["d","e","f"])
    cr.writerow(["a","b","c"])
    cr.writerow(["d","e","f"])

Whatever the python version, that will create a csv file without the infamous "blank lines".

The only drawback is that on Linux, this method would produce \r-free files, which is maybe not the standard (although files still opens properly in excel, no blank lines and still several lines :))

the problem persists on 3.6.2 (Just checked myself like I should have some time ago)

An alternative is to use a dictionary as arguments:

write_args = {"mode":"wb"} if bytes is str else {"mode":"w","newline":""}

(comparing bytes to str is one of the many ways to tell python 2 from python 3, in python 3 types are different, and it's very related to our current problem BTW).

Now we can pass those arguments with args unpacking:

with open("out.csv",**write_args) as f:
    cr = csv.writer(f,delimiter=";")

Answer 2

For both reading and writing csv files, I've found no better way either — however I would encapsulate into a separate function as shown below. The advantage being that the logic is all in one place instead of duplicated if it's needed more than once.

import csv
import sys

def open_csv(filename, mode='r'):
    """Open a csv file in proper mode depending on Python verion."""
    return(open(filename, mode=mode+'b') if sys.version_info[0] == 2 else
           open(filename, mode=mode, newline=''))

with open_csv('out.csv', 'w') as f:
    writer = csv.writer(f, delimiter=';')
    writer.writerow([1, 2, 3])
    writer.writerow(['a', 'b', 'c'])

The open_csv() utility could be simplified slightly by using the technique shown in @Jean-François Fabre's Dec 8, 2020 update to his answer to detect what version of Python is being used:

def open_csv(filename, mode='r'):
    """Open a csv file in proper mode depending on Python verion."""
    return(open(filename, mode=mode+'b') if bytes is str else
           open(filename, mode=mode, newline=''))