For log processing my application needs to read text files line by line.
First I used the function readLine() of BufferedReader but I read on the internet that BufferedReader is slow when reading files.
Afterwards I tried to use FileInputStream together with a FileChannel and MappedByteBuffer but in this case there's no function similar to readLine() so I search my text for a line-break and process it:
try {
FileInputStream f = new FileInputStream(file);
FileChannel ch = f.getChannel( );
MappedByteBuffer mb = ch.map(FileChannel.MapMode.READ_ONLY, 0L, ch.size());
byte[] bytes = new byte[1024];
int i = 0;
while (mb.hasRemaining()) {
byte get = mb.get();
if(get == '\n') {
if(ra.run(new String(bytes)))
cnt++;
for(int j = 0; j<=i; j++)
bytes[j] = 0;
i = 0;
}
else
bytes[i++] = get;
}
} catch(Exception ex) {
ex.printStackTrace();
}
I know this is probably not a good way to implement it but when I just read the text-file in bytes it is 3 times faster then using BufferedReader but calling new String(bytes)
creates a new String and makes the program even slower then when using a BufferedReader.
So I wanted to ask what is the fastest way to read a text-file line by line? Some say BufferedReader is the only solution to this problem.
P.S.: ra
is an instance of RunAutomaton from the dk.brics.Automaton library.
Java 8 has added a new method called lines() in the Files class which can be used to read a file line by line in Java. The beauty of this method is that it reads all lines from a file as Stream of String, which is populated lazily as the stream is consumed.
We can use java. io. BufferedReader readLine() method to read file line by line to String. This method returns null when end of file is reached.
The hasNextLine() method returns true if there is another line in the input of this scanner, but the scanner itself does not advance past any input or read any data at this point. To read the line and move on, we should use the nextLine() method.
Using the Java BufferedRedaer class is the most common and simple way to read a file line by line in Java. It belongs to java.io package. Java BufferedReader class provides readLine() method to read a file line by line.
Using plain BufferedReader I got 100+ MB/s. It is highly likely that the speed you can read the data from disk is your bottle neck, so how you do the reading won't make much difference.
BufferedReader is not the only solution, but it is fast enough for 99% of use cases, so why make things more complicated than they need to be?
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With