Is there a way to set up a Linux pipe to non-buffering or line-buffering?

Question

My program is controlling an external application on Linux, passing in input commands via a pipe to the external applications stdin, and reading output result via a pipe from the external applications stdout.

The problem is that writes to pipes are buffered by block, and not by line, and therefore delays occur before my app receives data output by the external application. The external application cannot be altered to add explicit fflush() calls.

When I set the external application to /bin/cat -n (it echoes back the input, with line numbers added), it works correctly, it seems, cat flushes after each line. The only way to force the external application to flush, is sending exit command to it; as it receives the command, it flushes, and all the answers appears on the stdout, just before exiting.

I'm pretty sure, that Unix pipes are appropiate solution for that kind of interprocess communication (pseudo server-client), but maybe I'm wrong.

(I've just copied some text from a similar question: Force another program's standard output to be unbuffered using Python)

Answer 1

Don't use a pipe. Use a pty instead. Pty's (pseudo-ttys) have the benefit of being line buffered if you want it, which provides you with simple framing for your data stream.

Answer 2

Using a PTY may be an overkill for the problem at hand (although it will work).

If the "target application" (the Delphi command-line utility) is dynamically linked, a possibly much simpler solution is to interpose (via LD_PRELOAD) a small library into the application. That library simply needs to implement isatty, and answer true (return 1) regardless of whether the output is going to a pipe or a terminal. You may wish to do that for all file descriptors, or just for STDOUT_FILENO.

Most UNIX implementations will call isatty to decide whether to do full buffering or line buffering for a given file descriptor.

Hmm, glibc doesn't. It calls __fxstat, and then only calls isatty if the status indicates that fd is going to a character device. So you'll need to interpose both __fxstat and isatty. More on library interposition here.