Python 3: Unbuffered vs Buffered Streams

Question

I have been using the following snippet to silence (redirect output from) C code called in my Python script:

from ctypes import CDLL, c_void_p
import os
import sys

# Code
class silence(object):
    def __init__(self, stdout=os.devnull):
        self.outfile = stdout

    def __enter__(self):
        # Flush
        sys.__stdout__.flush()

        # Save
        self.saved_stream = sys.stdout
        self.fd = sys.stdout.fileno()
        self.saved_fd = os.dup(self.fd)

        # Open the redirect
        self.new_stream = open(self.outfile, 'wb', 0)

        self.new_fd = self.new_stream.fileno()

        # Replace
        os.dup2(self.new_fd, self.fd)

    def __exit__(self, *args):
        # Flush
        self.saved_stream.flush()

        # Restore
        os.dup2(self.saved_fd, self.fd)
        sys.stdout = self.saved_stream

        # Clean up
        self.new_stream.close()
        os.close(self.saved_fd)


# Test case
libc = CDLL('libc.so.6')


# Silence!
with silence():
    libc.printf(b'Hello from C in silence\n')

The idea is to redirect the fd associated with stdout and replace it with one associated with an open null device. Unfortunately, it does not work as expected under Python 3:

$ python2.7 test.py
$ python3.3 -u test.py
$ python3.3 test.py
Hello from C in silence

Under Python 2.7 and 3.3 with unbuffered output it does work. I am unsure what the underlying cause is, however. Even if stdout is buffered the call to sys.saved_stream.flush() should end up calling fflush(stdout) at the C level (flushing the output to the null device).

What part of the Python 3 I/O model am I misunderstanding?

Answer 1

I'm not 100% sure I understand the Py3 I/O model either, but adding

    sys.stdout = os.fdopen(self.fd, 'wb', 0)

right after your assignment to self.fd fixes it for me in Python 3.4 (I was able to reproduce the problem in 3.4 before I added this statement).