Why do I get a java.nio.BufferUnderflowException in this Scala

Question

I was trying to do some scripting in Scala, to process some log files:

scala> import io.Source
import io.Source

scala> import java.io.File
import java.io.File

scala> val f = new File(".")
f: java.io.File = .

scala> for (l <- f.listFiles) {
 |   val src = Source.fromFile(l).getLines
 |   println( (0 /: src) { (i, line) => i + 1 } )
 | }
3658
java.nio.BufferUnderflowException
        at java.nio.Buffer.nextGetIndex(Unknown Source)
        at java.nio.HeapCharBuffer.get(Unknown Source)
        at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:86)
        at scala.io.BufferedSource$$anon$2.next(BufferedSource.scala:74)
        at scala.io.Source$$anon$6.next(Source.scala:307)
        at scala.io.Source$$anon$6.next(Source.scala:301)
        at scala.Iterator$cla...

Why do I get this java.nio.BufferUnderflowException?

NOTE - I'm processing 10 log files, each about 1MB in size

Answer 1

I got BufferUnderflowException exception when I opened a file with the wrong enconding. It contained illegal characters (according to the wrong encoding) and this misleading exception was thrown.

Answer 2

I'd also be interested as to exactly why this is happening but I'd guess it's to do with the fact that Source is an object (i.e. a singleton) and how it is gets transparently reset. You can fix the problem as follows:

for (l <- g.listFiles if !l.isDirectory) {
 | val src = Source.fromFile(l)
 | println( (0 /: src.getLines) { (i, line) => i + 1 } )
 | src.reset
 | }

The important bit is the reset - which should probably be in a try-finally block (although the isDirectory test is probably useful too)

Answer 3

This is essentially a restatement of Elazar's answer, but you will also get this exception if you try to read a binary file using scala.io.Source.fromFile.

I just ran into this (accidentally trying to read a .jpg with fromFile) due to a very stupid bug in something I wrote...