Why do I have to press Ctrl+D twice to close stdin?

Question

I have the following Python script that reads numbers and outputs an error if the input is not a number.

import fileinput
import sys
for line in (txt.strip() for txt in fileinput.input()):
    if not line.isdigit():
        sys.stderr.write("ERROR: not a number: %s\n" % line)

If I get the input from stdin, I have to press Ctrl + D twice to end the program. Why?

I only have to press Ctrl + D once when I run the Python interpreter by itself.

bash $ python test.py
1
2
foo
4
5
<Ctrl+D>
ERROR: not a number: foo
<Ctrl+D>
bash $

Answer 1

In Python 3, this was due to a bug in Python's standard I/O library. The bug was fixed in Python 3.3.

---

In a Unix terminal, typing Ctrl+D doesn't actually close the process's stdin. But typing either Enter or Ctrl+D does cause the OS read system call to return right away. So:

>>> sys.stdin.read(100)
xyzzy                       (I press Enter here)
                            (I press Ctrl+D once)
'xyzzy\n'
>>>

sys.stdin.read(100) is delegated to sys.stdin.buffer.read, which calls the system read() in a loop until either it accumulates the full requested amount of data; or the system read() returns 0 bytes; or an error occurs. (docs) (source)

Pressing Enter after the first line caused the system read() to return 6 bytes. sys.stdin.buffer.read called read() again to try to get more input. Then I pressed Ctrl+D, causing read() to return 0 bytes. At this point, sys.stdin.buffer.read gave up and returned just the 6 bytes it had collected earlier.

Note that the process still has my terminal on stdin, and I can still type stuff.

>>> sys.stdin.read()        (note I can still type stuff to python)
xyzzy                       (I press Enter)
                            (Press Ctrl+D again)
'xyzzy\n'

OK. This is the part that was busted when this question was originally asked. It works now. But prior to Python 3.3, there was a bug.

The bug was a little complicated --- basically the problem was that two separate layers were doing the same work. BufferedReader.read() was written to call self.raw.read() repeatedly until it returned 0 bytes. However, the raw method, FileIO.read(), performed a loop-until-zero-bytes of its own. So the first time you press Ctrl+D in a Python with this bug, it would cause FileIO.read() to return 6 bytes to BufferedReader.read(), which would then immediately call self.raw.read() again. The second Ctrl+D would cause that to return 0 bytes, and then BufferedReader.read() would finally exit.

This explanation is unfortunately much longer than my previous one, but it has the virtue of being correct. Bugs are like that...