Efficient method to read String lines from file

Question

Assuming I have 15GB log records file, and I would like to iterate over \n terminated lines from this file. What java standard lib / 3rd parties provide clean interface for this operation.

Please note that I'm seeking for an NIO based solution, preferablly using Memory Mapped file access method, as demoed by this question How do I create a Java string from the contents of a file? would would be a perfect solution had it not loaded the whole byte buffer into memory before returning a new String() instance of the buffer. This approach does not work in this case because of the size of the input.

Thank you,
Maxim.

Answer 1

Have you considered using a BufferedReader? From the documentation:

Reads text from a character-input stream, buffering characters so as to provide for the efficient reading of characters, arrays, and lines.

It has a clean interface for getting \n-terminated strings (BufferedReader.readLine()) and should be fairly efficient since it is buffered.