Reading Input and Error Streams Concurrently using BufferedReaders Hangs

Question

First off let me apologize to the SO community for coming to you with something that ought to be so trivial. But I've been at this all day and I'm at the end of my rope.

There is a section of my program that needs pull text from an input stream and an error stream from a process that is launched using Runtime.getrunTime().exec() and pass it through to standard input and output in an orderly manner. I have a function that near as I can tell should work. But it seems to be getting caught in a catch-22 where it's waiting for the stream to report ready - but the stream has finished and is not reporting. I'm baffled. I can't think of another way to do this that fits my constraints and I'm rather skeptical that such a catch-22 can exist.

Here is my code:

private void forwardStreamtoStd(InputStream in, InputStream err) 
throws IOException {
    int c = -1;
    BufferedReader inReader = new BufferedReader(
        new InputStreamReader(in, "US-ASCII"));
    BufferedReader errReader = new BufferedReader(
        new InputStreamReader(err, "US-ASCII"));
    boolean inFinished = false, errFinished = false;

    try {
        System.out.println("Begin stream read loop...");
        while (!inFinished && !errFinished) {
        if (!inFinished) {
            while (inReader.ready()) {
                if ((c = inReader.read()) == -1) {
                    inFinished = true;
                } 
                else {
                    System.out.print((char) c);
                }
            }
        }

        if (!errFinished) {
            while (errReader.ready()) {
                if ((c = errReader.read()) == -1) {
                    errFinished = true;
                } 
                else {
                     System.err.print((char) c);
                }
            }
        }
        }
        System.out.println("End stream read loop.");
    } 
    catch (IOException e) {
        throw e;
    } 
    finally {
        errReader.close();
        inReader.close();
    }
}

The problem seems to be that the reading loops are waiting for the streams to report ready, and as a result aren't seeing the -1 returned by read telling them that it's time to quit. I'm trying to avoid having either stream blocking, so that I can pull from both in turn when they are prepared. However, how can I catch the process's end of stream? Am I missing something? Shouldn't read report that it's read when it has an end of stream -1? The processes are finishing, and so their streams should be dying. What am I doing wrong here?

Answer 1

There are two more possibilities:

• Use the ProcessBuilder and invoke redirectErrorStream(true) to join the two streams and you need to read one stream. I have an example here.
• In JDK7, you could call the inheritIO() to automatically forward everything

Edit On the second guess, it seems the ready() call is misleading your program. Try this:

private void forwardStreamtoStd(InputStream in, InputStream err) 
throws IOException {
    int c = -1;
    BufferedReader inReader = new BufferedReader(
        new InputStreamReader(in, "US-ASCII"));
    BufferedReader errReader = new BufferedReader(
        new InputStreamReader(err, "US-ASCII"));
    boolean inFinished = false, errFinished = false;

    try {
        System.out.println("Begin stream read loop...");
        if (!inFinished) {
            while ((c = inReader.read()) != -1) {
                    System.out.print((char) c);
            }
            inFinished = true;
        }

        if (!errFinished) {
            while ((c = errReader.read()) != -1) {
                System.err.print((char) c);
            }
            errFinished = true;
        }
        System.out.println("End stream read loop.");
    } 
    catch (IOException e) {
        throw e;
    } 
    finally {
        errReader.close();
        inReader.close();
    }
}

Or better yet, leave off the BufferedReader if you don't plan any extra transformation:

private void createReader(final InputStream in, final OutputStream out) {
    new Thread() {
        public void run() {
            try {
                int c = 0;
                while ((c = in.read()) != -1) {
                    out.write(c);
                }
            } catch (IOException ex) {
                ex.printStackTrace();
            } finally {
                in.close();
            }
        }
    }.start();
}
private void forwardStreamtoStd(InputStream in, InputStream err) 
throws IOException {
    createReader(in, System.out);
    createReader(err, System.err);
}

Answer 2

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4090471

The solution I've always used is to create a separate thread to read one of the streams, join on the thread when the main thread finishes reading, and then waitFor the process.

Answer 3

It's essential to consume the 2 streams concurrently, to prevent blocking. See this article for more info, and in particular note the StreamGobbler mechanism that captures stdout/err in separate threads.