Java : Read last n lines of a HUGE file

Question

I want to read the last n lines of a very big file without reading the whole file into any buffer/memory area using Java.

I looked around the JDK APIs and Apache Commons I/O and am not able to locate one which is suitable for this purpose.

I was thinking of the way tail or less does it in UNIX. I don't think they load the entire file and then show the last few lines of the file. There should be similar way to do the same in Java too.

Answer 1

I found it the simplest way to do by using ReversedLinesFileReader from apache commons-io api. This method will give you the line from bottom to top of a file and you can specify n_lines value to specify the number of line.

import org.apache.commons.io.input.ReversedLinesFileReader;   File file = new File("D:\\file_name.xml"); int n_lines = 10; int counter = 0;  ReversedLinesFileReader object = new ReversedLinesFileReader(file); while(counter < n_lines) {     System.out.println(object.readLine());     counter++; } 

Answer 2

If you use a RandomAccessFile, you can use length and seek to get to a specific point near the end of the file and then read forward from there.

If you find there weren't enough lines, back up from that point and try again. Once you've figured out where the Nth last line begins, you can seek to there and just read-and-print.

An initial best-guess assumption can be made based on your data properties. For example, if it's a text file, it's possible the line lengths won't exceed an average of 132 so, to get the last five lines, start 660 characters before the end. Then, if you were wrong, try again at 1320 (you can even use what you learned from the last 660 characters to adjust that - example: if those 660 characters were just three lines, the next try could be 660 / 3 * 5, plus maybe a bit extra just in case).