Run a process asynchronously and read from stdout and stderr

Question

I have some code that runs a process and reads from the stdout and stderr asynchronously and then handles when the process completes. It looks something like this:

Process process = builder.start();

    Thread outThread = new Thread(() -> {
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
            // Read stream here
        } catch (Exception e) {
        }
    });

    Thread errThread = new Thread(() -> {
      try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {
        // Read stream here
      } catch (Exception e) {
      }
    });

    outThread.start();
    errThread.start();

    new Thread(() -> {
      int exitCode = -1;
      try {
        exitCode = process.waitFor();
        outThread.join();
        errThread.join();
      } catch (Exception e) {
      }

    // Process completed and read all stdout and stderr here
    }).start();

My issue is with the fact that I am using 3 threads to achieve this asynchronous "run-and-get-output" task - I don't know why, but I feel it doesn't feel right using 3 threads. I could allocate the threads out of a thread pool, but that would still be blocking those threads.

Is there anything I can do, maybe with NIO, to reduce this to fewer (1?) thread? Anything I can think of will be constantly spinning a thread (unless I add a few sleeps), which I don't really want to do either...

NOTE: I do need to read as I go (rather than when the process has stopped) and I do need to separate stdin from stderr so can't do a redirect.

Answer 1

Assuming you don't mind the input and error streams to be merged, you could only use one thread with:

builder.redirectErrorStream(true); //merge input and error streams
Process process = builder.start();

Thread singleThread = new Thread(() -> {
  int exitCode = -1;
  //read from the merged stream
  try (BufferedReader reader = 
              new BufferedReader(new InputStreamReader(process.getInputStream()))) {
    String line;
    //read until the stream is exhausted, meaning the process has terminated
    while ((line = reader.readLine()) != null) {
      System.out.println(line); //use the output here
    }
    //get the exit code if required
    exitCode = process.waitFor();
  } catch (Exception e) { }
}).start();

Answer 2

Since you've specified that you need to read the output as you go, there is no non-multi-threaded solution.

You can reduce the number of threads to one beyond your main thread though:

Process process = builder.start();
Thread errThread = new Thread(() -> {
    try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getErrorStream()))) {
      // Read stream here
    } catch (Exception e) {
    }
});
errThread.start();

try (BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
        // Read stream here
} catch (Exception e) {
}
// we got an end of file, so there can't be any more input.  Now we need to wait for stderr/process exit.

int exitCode = -1;
try {
    exitCode = process.waitFor();
    errThread.join();
} catch (Exception e) {
}

// Process completed

If you truely don't need to deal with the error/output until after the process ends, you can simplify it a bit and only use your main thread like this:

    File stderrFile = File.createTempFile("tmpErr", "out");
    File stdoutFile = File.createTempFile("tmpStd", "out");
    try {
        ProcessBuilder builder = new ProcessBuilder("ls /tmp");
        Process p = builder.start();
        int exitCode = -1;
        boolean done = false;
        while (!done) {
            try {
                exitCode = p.waitFor();
                done = true;
            } catch (InterruptedException ie) {
                System.out.println("Interrupted waiting for process to exit.");
            }
        }
        BufferedReader err = new BufferedReader(new FileReader(stderrFile));
        BufferedReader in = new BufferedReader(new FileReader(stdoutFile));
        ....
    } finally {
        stderrFile.delete();
        stdoutFile.delete();
    }

This is probably not a good idea if you generate a lot of output from the process you are calling as it could run out of disk space... but it'll likely be slightly faster since it doesn't have to spin up another Thread.