java: ProcessBuilder makes a memory hog

Question

I have some issues regarding ProcessBuilder. The program is basically a simple wrapper invoking a command line script.

When running the script on its own via the terminal, the memory consumption stays below 2G. When running the script via the java wrapper, the memory consumption explodes and even 8G is quickly filled up, resulting in out-of-memory errors.

The code to launch the process is simply:

public static int execute(String command) throws IOException
 {
  System.out.println("Executing: " + command);

  ProcessBuilder pb = new ProcessBuilder(command.split(" +"));
  Process p = pb.start();

  // display any output in stderr or stdout  
  StreamConsumer stderr = new StreamConsumer(p.getErrorStream(), "stderr");
  StreamConsumer stdout = new StreamConsumer(p.getInputStream(), "stdout");
  new Thread(stderr).start();
  new Thread(stdout).start();

  try {
   return p.waitFor();
  } catch (InterruptedException e) {
   throw new RuntimeException(e); 
  }
 }

The StreamConsumer class is simply a class which consumes the stdout/stderr streams and display them on the console.

...the question is: why on earth does the memory consumption explode?

Regards,
Arnaud

Edit:

• Whether I use ProcessBuilder or Runtime.getRuntime.exec(...), the result is the same.
• The memory bursts tend to appear during unix 'sort' invoked by the shell script called:

sort big-text-file > big-text-file.sorted

Edit 2 on request of Jim Garrison:

Ok, here is the StreamConsumer class which I omitted because it is rather simple:

class StreamConsumer implements Runnable
{
    InputStream stream;
    String descr;

    StreamConsumer(InputStream stream, String descr) {
        this.stream = stream;
        this.descr = descr;
    }

    @Override
    public void run()
    {
        String line;

        BufferedReader  brCleanUp = 
            new BufferedReader (new InputStreamReader (stream));

        try {
            while ((line = brCleanUp.readLine ()) != null)
                System.out.println ("[" + descr + "] " + line);
             brCleanUp.close();
        } catch (IOException e) {
            // TODO: handle exception
        }
    }
}

Answer 1

if you change your command like this : sort -o big-text-file.sorted big-text-file

is it always the same ?