C fork/exec with non-blocking pipe IO

Question

This seems to be a fairly common thing to do, and I've managed to teach myself everything that I need to make it work, except that I now have a single problem, which is defying my troubleshooting.

int nonBlockingPOpen(char *const argv[]){
    int inpipe;
    pid_t pid;
    /* open both ends of pipe nonblockingly */
    pid = fork();

    switch(pid){
        case 0:         /*child*/
            sleep(1); /*child should open after parent has open for reading*/

            /*redirect stdout to opened pipe*/
            int outpipe = open("./fifo", O_WRONLY);
            /*SHOULD BLOCK UNTIL MAIN PROCESS OPENS FOR WRITING*/
            dup2(outpipe, 1);
            fcntl(1, F_SETFL, fcntl(1, F_GETFL) | O_NONBLOCK);

            printf("HELLO WORLD I AM A CHILD PROCESS\n");
            /*This seems to be written to the pipe immediately, blocking or not.*/
            execvp(*argv, argv);
            /*All output from this program, which outputs "one" sleeps for 1 second
             *outputs "two" sleeps for a second, etc, is captured only after the
             *exec'd program exits!
             */
            break;

        default:        /*parent*/
            inpipe = open("./fifo", O_RDONLY | O_NONBLOCK);
            sleep(2);
            /*no need to do anything special here*/
            break;
    }

    return inpipe;
}

Why won't the child process write its stdout to the pipe each time a line is generated? Is there something I'm missing in the way execvp or dup2 work? I'm aware that my approach to all this is a bit strange, but I can't find another way to capture output of closed-source binaries programatically.

Answer 1

I would guess you only get the exec'd program's output after it exits because it does not flush after each message. If so, there is nothing you can do from the outside.

I am not quite sure how this is supposed to relate to the choice between blocking and nonblocking I/O in your question. A non-blocking write may fail completely or partially: instead of blocking the program until room is available in the pipe, the call returns immediately and says that it was not able to write everything it should have. Non-blocking I/O neither makes the buffer larger nor forces output to be flushed, and it may be badly supported by some programs.

You cannot force the binary-only program that you are exec'ing to flush. If you thought that non-blocking I/O was a solution to that problem, sorry, but I'm afraid it is quite orthogonal.

EDIT: Well, if the exec'd program only uses the buffering provided by libc (does not implement its own) and is dynamically linked, you could force it to flush by linking it against a modified libc that flushes every write. This would be a desperate measure. to try only if everything else failed.

Answer 2

When a process is started (via execvp() in your example), the behaviour of standard output depends on whether the output device is a terminal or not. If it is not (and a FIFO is not a terminal), then the output will be fully buffered, rather than line buffered. There is nothing you can do about that; the (Standard) C library does that.

If you really want to make it work line buffered, then you will have to provide the program with a pseudo-terminal as its standard output. That gets into interesting realms - pseudo-terminals or ptys are not all that easy to handle. For the POSIX functions, see:

• grantpt() - grant access to the slave pseudo-terminal device
• posix_openpt() - open a pseudo-terminal device
• ptsname() - get name of the slave pseudo-terminal device
• unlockpt() - unlock a pseudo-terminal master/slave pair