does Python for windows ever insert '\r\n' when told to insert '\n'?

Question

I use a PC at home and a Mac at work. I've never had any problems with line breaks in python script or their outputs, but whenever i send something to my boss i get an angry e-mail back about windows line breaks in it.

The most recent was the output of a python script where i'd told it to end every line with '\n', but on closer inspection (on my Mac at work) it seems that each line did in fact end with '\r\n'.

What's going on, and how do i stop it? I used to run all my scripts in a Linux virtual machine at home, but i found that was too slow and fiddly, surely there's a simpler fix?

Answer 1

This is because you have files opened in text mode and Python is normalizing the newlines in accordance with the platform you're using (Windows used \r\n and Linux just uses \n). You need to open files in binary mode like this:

f = open("myfile.txt","wb")

It does the same thing in reverse when you read in files (\r\n will be replaced by \n) unless you also specify binary mode:

f = open("myfile.txt", "rb")

Answer 2

The behavior you are seeing is not python-specific. It comes from the buffered file-handling functions in the C standard library that underlies python and other high level languages. Unless told not to, it will convert newline characters to the current platform's native text file line break sequence when writing, and do the reverse when reading. See the documentation for fopen() on your local system for details. On Windows, it means \n will be converted to \r\n on writes.

The python docs mention newline conversion and other open() mode options here.

One solution would be to use open("filename", "wb") instead of open("filename", "w") when opening the output file in the first place. That will avoid the automatic newline conversion. It ought to solve the problem for your boss, so long as your boss is using some form of unix (including OSX). Unfortunately, it will also mean that some Windows text editors (e.g. notepad?) will present your file strangely:

Windows acts like a teletype
                            when it sees new lines
                                                  without carriage returns.

Another approach would be to convert your files as needed before sending them to someone who doesn't use Windows. Various conversion programs exist for this purpose, such as dos2unix and flip.