Getting another program's output as input on the fly

Question

I've two programs I'm using in this way:

$ c_program | python_program.py

c_program prints something using printf() and python_program.py reads using sys.stdin.readline()

I'd like to make the python_program.py process c_program's output as it prints, immediately, so that it can print its own current output. Unfortunately python_program.py gets its input only after c_program ends.

How can I solve this?

Answer 1

Just set stdout to be line buffered at the beginning of your C program (before performing any output), like this:

#include <stdio.h>
setvbuf(stdout, NULL, _IOLBF, 0);

or

#include <stdio.h>
setlinebuf(stdout);

Either one will work on Linux, but setvbuf is part of the C standard so it will work on more systems.

By default stdout will be block buffered for a pipe or file, or line buffered for a terminal. Since stdout is a pipe in this case, the default will be block buffered. If it is block buffered then the buffer will be flushed when it is full, or when you call fflush(stdout). If it is line buffered then it will be flushed automatically after each line.

Answer 2

What you need is for your C program to call fflush(stdout) after every line. For example, with the GNU grep tool, you can invoke the option '--line-buffered', which causes this behavior. See fflush.

Answer 3

If you can modify your C program, you've already received your answer but i thought i'd include a solution for those that can't/won't modify code.

expect has an example script called unbuffer that will do the trick.

Answer 4

You may want to try flushing the stdout stream in the cpp program.