all but the last item from a Scala Iterator (a.k.a. Iterator.init)

Question

I have a number of large files where I want to process all but the last line in each file. If the files were small, I could just convert to a TraversableLike and use the "init" method, e.g.:

lines.toList.init

But the files are large so I need to keep things as an iterator. Is there a simple way to get something like "init" on an Iterator? I'm considering the following, but I'm not confident it will always work:

lines.takeWhile(_ => lines.hasNext)

Answer 1

You're relying on takeWhile grabbing the next line first, and then checking to see if there is another line. It does work this way in the default iterator. And for any particular iterator, I expect that if it ever works, it should always work.

If it ever fails, you'll need to cache the result. You could make your own iterator that does this, or, more easily (if this isn't urgently performance-critical code):

lines.sliding(2).map(_.head)

Test this out as follows:

List(1,2,3,4,5).iterator.sliding(2).map(_.head).toList

You do not have to rely upon takeWhile or any other method behaving any particular way; if sliding is not broken, it will have the behavior above.