Java.io : Performance Tuning

Question

I am having a file of around 4MB, the file is an ascii file containing normal keyboard characters only. I tried many classes in java.io package to read the file contents as string. Reading them character by character (using FileReader and BufferedReader) takes approximately 40 seconds, reading the content using java.nio package (FileChannel and ByteBuffer) takes approximately 25 seconds. This is from my knowledge a little bit greater amount of time. Does someone knows any way to reduce this time consumption to somewhat around 10 seconds? Even solutions like creating file reader using C and calling from java will do. I used the below snippet to read the 4 MB file in 22 seconds-

public static String getContents(File file) {
    try {
        if (!file.exists() && !file.isFile()) {
            return null;
        }
        FileInputStream in = new FileInputStream(file);
        FileChannel ch = in.getChannel();
        ByteBuffer buf = ByteBuffer.allocateDirect(512);            
        Charset cs = Charset.forName("ASCII");          
        StringBuilder sb = new StringBuilder();
        int rd;
        while ((rd = ch.read(buf)) != -1) {
            buf.rewind();
            CharBuffer chbuf = cs.decode(buf);
            for (int i = 0; i < chbuf.length(); i++) {
                sb.append(chbuf.get());
            }
            buf.clear();
        }
        String contents = sb.toString();
        System.out.println("File Contents:\n"+contents);
        return contents;
    } catch (Exception exception) {
        System.out.println("Error:\n" + exception.getMessage());
        return null;
    }
}

Answer 1

I can't imagine what your hardware could be but it should take less than 0.1 seconds for a 4 MB file.

A fast way to read the file all at once is to read it into a byte[]

public static String readFileAsString(File file) {
    try {
        DataInputStream in = new DataInputStream(FileInputStream(file));
        byte[] bytes = new byte[(int) file.length()];
        in.readFully(bytes);
        in.close();
        return new String(bytes, 0); // ASCII text only.

    } catch (FileNotFoundException e) {
        return null;
    } catch (IOException e) {
        System.out.println("Error:\n" + e.getMessage());
        return null;
    }
}

public static void main(String... args) throws IOException {
    File tmp = File.createTempFile("deleteme", "txt");
    tmp.deleteOnExit();

    byte[] bytes = new byte[4 * 1024 * 1024];
    Arrays.fill(bytes, (byte) 'a');
    FileOutputStream fos = new FileOutputStream(tmp);
    fos.write(bytes);
    fos.close();

    long start = System.nanoTime();
    String s = readFileAsString(tmp);
    long time = System.nanoTime() - start;
    System.out.printf("Took %.3f seconds to read a file with %,d bytes%n",
            time / 1e9, s.length());
}

prints

Took 0.026 seconds to read a file with 4,194,304 bytes

If you want to read the file even faster, I suggest using a memory mapped file as it will take less than 10 milli-seconds, but that is over kill in this case.

Answer 2

1. There is no benefit in using direct byte buffers here.
2. Your buffer size of 512 is too small. Use at least 4096.
3. There is no real benefit to using NIO here. As this is text, I would use a BufferedReader.
4. Your basic objective of reading the entire file into memory is flawed. It will not scale and it already uses excessive amounts of memory. You should devise a strategy for handling the file a line at a time.