Surprising timing with conversion from byte[] to String

Question

I'm using a DataInputStream to read a byte array from a file and convert to a string. Here is the original code. Note that dis is a DataInputStream on a BufferedInputStream on a GZipInputStream on a FileInputStream.

// class definition
var byteBuffer = Array[Byte](0)

...

// Get the payload
if (contentLength > byteBuffer.length) {
  println("resize")
  byteBuffer = new Array[Byte](contentLength, "UTF-8")
}
dis.read(byteBuffer, 0, contentLength)

new String(byteBuffer)

This code and the surrounding processing is slow. I only process 80 document per second. A small change increases the speed dramatically.

// Get the payload
val byteBuffer = new Array[Byte](contentLength, "UTF-8")
dis.read(byteBuffer, 0, contentLength)

new String(byteBuffer)

Now I am processing nearly 300 documents per second. It makes little sense to me why allocating the array each time should provide a substantial speed benefit, even after digging into the decoding code a little bit. Any ideas?

The val/var change is irrelevant. It get the same speed boost if I just remove the conditional.

Answer 1

In the second case, you make a string of the right size. In the first case, your strings are all as large as the largest string you've previously created.

You probably have later processing code that makes you not notice this difference?