How to write the contents of a Scala stream to a file?

Question

I have a Scala stream of bytes that I'd like to write to a file. The stream has too much data to buffer all of it memory.

As a first attempt, I created an InputStream similar to this:

class MyInputStream(data: Stream[Byte]) extends InputStream {
  private val iterator = data.iterator
  override def read(): Int = if (iterator.hasNext) iterator.next else -1
}

Then I use Apache Commons to write the file:

val source = new MyInputStream(dataStream)
val target = new FileOutputStream(file)
try {
  IOUtils.copy(source, target)
} finally {
  target.close
}

This works, but I'm not too happy with the performance. I'm guessing that calling MyInputStream.read for every byte introduces a lot of overhead. Is there a better way?

Answer 1

You might (or might not!) be mistaken that the read side is the source of your performance troubles. It could be the fact that you are using an unbuffered FileOutputStream(...), forcing a separate system call for every byte written.

Here's my take, quick 'n simple:

def writeBytes( data : Stream[Byte], file : File ) = {
  val target = new BufferedOutputStream( new FileOutputStream(file) )
  try data.foreach( target.write(_) ) finally target.close
}