Python in raw mode stdin print adds spaces

Question

I needed to switch the standard input to non-buffered mode in Python, so that I can read single characters off it. I managed to get it working, but now the standard output is broken: somehow it seems like after the newline character, some space characters are emitted, zero on the first line, 3 on the second, 6 on the third, etc, like this:

ASD
   ASD
      ASD

Operating system is Ubuntu Linux 12.04, 64-bit edition, Python version is 3.2.3.

How can I rid myself from this behavior?

Below is the code I've used:

import sys
import tty
import termios

fd = sys.stdin.fileno()
old_settings = termios.tcgetattr(fd)
tty.setraw(sys.stdin)

for i in range(0, 10):
    print("ASD")

termios.tcsetattr(fd, termios.TCSADRAIN, old_settings)

Answer 1

The problem you're experiencing is the difference between 'raw', 'cooked', and 'cbreak' modes. And these modes are modes of the kernel level terminal driver, not modes of your application code or the standard library or anything else in userspace. This is the old-school Unix way of referring to these. Posix has replaced them with a much more fine-grained set of attributes, though the Posix attributes are typically flipped in concert with helper functions in a way that mimics the old 'raw', 'cooked' and 'cbreak' modes.

In cooked mode, the terminal driver itself has primitive line editing functionality builtin. It handles backspace, word erase (basically backspace a whole word at once) and similar things. Nothing sophisticated like handling arrow keys or history or anything like that. Very primitive. In this mode, your program never sees anything from the terminal until the end-of-line (eol) character is sent, and then your program gets a whole line, and the line ending is translated to Unix standard \n regardless of what the terminal actually does. Also, as part of this, the terminal driver echoes typed characters back to the terminal so the user can see what they're typing.

In 'cooked' mode, the kernel level terminal driver also does some output translation. And part of that is turning \n into \r\n if needed.

Also, in 'cooked' mode the terminal driver handles special characters like Control-C (sends a SIGINT to the controlling process group (translated by CPython into a KeyboardInterrupt exception)) and Control-Z (sends a SIGTSTP (like a SIGSTOP, but can be caught) to the controlling process group).

In 'cbreak' mode, line editing is no longer done. The terminal driver gives each character (or short character sequence, like the escape sequence for an arrow key) to the program immediately. These characters are not echoed to the screen, and so unless your program then prints them the user won't see them. The terminal driver though still handles special characters like Control-C and Control-Z, though it ceases to handle line editing characters like backspace or the word-erase character (typically Control-W). Also, some output processing is still done, so the driver turns a \n into a \r\n.

In 'raw' mode, no processing is done on either input or output. No special character handling, no echoing, no transforming \n into \r\n, no handling for Control-Z, nothing. It's up to the program that puts the terminal in raw mode to do it all.

Now, you are setting the attributes for sys.stdin so you may think this shouldn't affect sys.stdout. But, in fact, both of your file descriptors lead to the exact same 'instance' of a terminal driver. And it's the settings for the terminal driver that determine what happens. So it doesn't matter if you change those settings through sys.stdin, sys.stdout, or even sys.stderr, all change the same underlying terminal driver instance and they affect all the others.

This, of course, is not true of file descriptors that have been redirected by the shell before your program is launched.

As a side note, you can use the stty -a on the command line to see a full read out of all of these flags (including which control characters result in which signals in cooked and cbreak modes).

Answer 2

The Google brought me here when I was searching for an answer to this same question. The clue shared by halex of no carriage return aided my search for the truth. I found my answers in a post on Chris's Wiki: https://utcc.utoronto.ca/~cks/space/blog/unix/CBreakAndRaw which lead me to read the source of tty.py here: https://hg.python.org/cpython/file/618ea5612e83/Lib/tty.py Which brought me to the conclusion that if the goal is to read single characters, instead of:

tty.setraw()

Use:

tty.setcbreak()

Answer 3

Looks like you are only doing a line feed but no carriage return. Change your print to

print("ASD", end="\r\n")