Streaming pipes in Python

Question

I'm trying to convert the output of vmstat into a CSV file using Python, so I use something like this to convert to CSV and add the date and time as coloumns:

vmstat 5 | python myscript.py >> vmstat.log

The problem I'm having is it blocks while trying to iterate sys.stdin. It seems like the input buffer doesn't get flushed. I don't want to endlessly loop around and burn processor time as I'm trying to measure this. Here's a simple demonstration which blocks on line 3:

import sys

for line in sys.stdin:
    sys.stdout.write(line)
    sys.stdout.flush()

Is there an easy way to access the stream immediately like grep does, without pausing while the input buffer fills up?

Answer 1

VMstat 5,does not close the stdout, so the python buffer is still waiting for more data.

Use this instead:

for line in iter(sys.stdin.readline, ""):
    print line