Does scala.sys.process block on stream processing threads?

Question

The Scala external process library, scala.sys.process, allows to run an external process (e.g. ls) and attach some processing functions to stdout and stderr. For example:

Seq("ls" ".").run(new ProcessIO(os => os.close(), is => f(is), is => g(is))).exitValue()

This runs ls . and runs some functions f and g on its stdout and stderr. Documentation says that f and g are executed in separate threads.

My question is, will .exitValue() block just until ls exits, or until not only ls exits but also those threads which run f and g are done?

Answer 1

A simple experiment:

def doStuff(sleepFor: Long): Unit = {
  val startTime = System.currentTimeMillis
  val result = Seq("ls", ".").run(
    new ProcessIO(
      { os: OutputStream => os.close() },
      { is: InputStream => Thread.sleep(sleepFor); is.close() },
      { is: InputStream => Thread.sleep(sleepFor); is.close() }
    )
  ).exitValue
  val finishTime = System.currentTimeMillis
  println(s"Result = $result, took ${ finishTime - startTime }ms")
}

When invoked, if we see that it took at least around sleepForms, it almost certainly blocked until the functions on standard out and standard error completed (especially if this holds for two very different values of sleepFor).

In an SBT console:

scala> doStuff(1000)
Result = 0, took 1002ms

scala> doStuff(10000)
Result = 0, took 10002ms

Experimentally verified that .run().exitValue blocks until the functions on input and output complete.

Tracing through the source (as of 2.13, but earlier versions should be similar), we see that a Seq[String] ultimately creates a Simple ProcessBuilder, which when run leads to a SimpleProcess:

override def exitValue() = {
  try p.waitFor()                   // wait for the process to terminate
  finally interrupt()
  outputThreads foreach (_.join())  // this ensures that all output is complete before returning (waitFor does not ensure this)

  p.exitValue()

The standard out/error processor threads are in outputThreads and are joined, which blocks until both threads are complete.

Side note: scala.sys.process is a pretty strong contender for most archaic API in the standard library, given that it predates the introduction of the Future API and the functions which result in a stream/lazy list of output lines encode the exit code as a bare RuntimeException where the message has to be parsed to extract the exit code (rather than a distinguished exception or even better a stream of Either[ExitWithErrorException, String]).