Is it possible to read the Process stdout InputStream into an NIO ByteBuffer?

Question

Is it possible to use NIO to process the stdout from a Process? I have it working with java.io, but this is something of an exercise to learn a bit more about NIO and to explore the possibility of performance improvements.

Basically I want to stream a large volume of text from stdout into a buffer as fast as possible without blocking, and then process the contents of that buffer later. The trouble is, I can't seem to figure out the right voodoo to make it work with NIO. This is where I am right now:

ProcessBuilder pb = new ProcessBuilder( ... );
Process p = pb.start();
stdout = new StreamConsumer(p.getInputStream());
new Thread(stdout).start();
// other stuff omitted for brevity

The class StreamConsumer looks like this:

class StreamConsumer implements Runnable
{
  private InputStream is;

  public StreamConsumer(InputStream is)
  {
    this.is = is;
  }

  public void run()
  {
    try
    {
      ReadableByteChannel source = Channels.newChannel(is);

      // Is it possible get a channel to a ByteBuffer 
      // or MappedByteBuffer here?
      WritableByteChannel destination = ??;
      ByteBuffer buffer = ByteBuffer.allocateDirect(128 * 1024);

      while (source.read(buffer) != -1)
      {
        buffer.flip();
        while (buffer.hasRemaining())
        {
          destination.write(buffer);
        }
        buffer.clear();
      }

      source.close();
      destination.close();
    }
    catch (IOException e)
    {
      e.printStackTrace();
    }
  }
}

Answer 1

I have created an open source library that allows non-blocking I/O between java and your child processes. The library provides an event-driven callback model. It depends on the JNA library to use platform-specific native APIs, such as epoll on Linux, kqueue/kevent on MacOS X, or IO Completion Ports on Windows.

The project is called NuProcess and can be found here:

https://github.com/brettwooldridge/NuProcess

Answer 2

Believe it or not, I think the writable byte channel you want is

ByteArrayOutputStream ostream = new ByteArrayOutputStream(<some large number>);
WritableByteChannel destination = Channels.newChannel(ostream);

Then when done

ostream.toByteArray() 

contains the bytes to process. Or, if you want a byte buffer,

ByteBuffer.wrap(ostream.toByteArray())

I don't see here how you get the output outside the runnable, but I suspect your original code had that. Otherwise you might want the StreamConsumer to be a Callable<ByteBuffer>.

Answer 3

you might want the StreamConsumer to be a Callable.

Another non blocking option to try might be to use Guava's ListenableFuture giving you success and failure callbacks without interpreting errors of your own.