What is os.linesep for?

Question

Python's os module contains a value for a platform specific line separating string, but the docs explicitly say not to use it when writing to a file:

Do not use os.linesep as a line terminator when writing files opened in text mode (the default); use a single '\n' instead, on all platforms.

Docs

Previous questions have explored why you shouldn't use it in this context, but then what context is it useful for? When should you use the line separator, and for what?

Answer 1

the docs explicitly say not to use it when writing to a file

This is not exact, the doc says not to used it in text mode.

The os.linesep is used when you want to iterate through the lines of a text file. The internal scanner recognise the os.linesep and replace it by a single "\n".

For illustration, we write a binary file which contains 3 lines separated by "\r\n" (Windows delimiter):

import io  filename = "text.txt"  content = b'line1\r\nline2\r\nline3' with io.open(filename, mode="wb") as fd:     fd.write(content) 

The content of the binary file is:

with io.open(filename, mode="rb") as fd:     for line in fd:         print(repr(line)) 

NB: I used the "rb" mode to read the file as a binary file.

I get:

b'line1\r\n' b'line2\r\n' b'line3' 

If I read the content of the file using the text mode, like this:

with io.open(filename, mode="r", encoding="ascii") as fd:     for line in fd:         print(repr(line)) 

I get:

'line1\n' 'line2\n' 'line3' 

The delimiter is replaced by "\n".

The os.linesep is also used in write mode: any "\n" character is converted to the system default line separator: "\r\n" on Windows, "\n" on POSIX, etc.

With the io.open function you can force the line separator to whatever you want.

Example: how to write a Windows text file:

with io.open(filename, mode="w", encoding="ascii", newline="\r\n") as fd:     fd.write("one\ntwo\nthree\n") 

If you read this file in text mode like this:

with io.open(filename, mode="rb") as fd:     content = fd.read()     print(repr(content)) 

You get:

b'one\r\ntwo\r\nthree\r\n' 

Answer 2

As you know, reading and writing files in text mode in python converts the platform specific line separator to '\n' and vice versa. But if you would read a file in binary mode, no conversion takes place. Then you can explicitly convert the line endings using string.replace(os.linesep, '\n'). This can be useful if a file (or stream or whatever) contains a combination of binary and text data.