Opening a large file in Java is very slow

Question

I have a large (12GB) file and I need to extract small pieces of data (a few kilobytes each) from it, using Java. Seeking and reading the data, once the file is open, is very fast, but opening the file itself takes a long time - about 90 seconds. Is there a way to speed up the open file operation in Java?

To clarify, I've tried the following options to open and read a file:

new FileInputStream(file); 
new RandomAccessFile(file, "r");
Files.newByteChannel(path, StandardOpenOption.READ);

Each one of these yielded similar results.

Answer 1

From the commments: To be specific, the problem is that Java's open file operation triggers the OS operation that runs the virus scan, and the solution is to add Java to the list of trusted processes

Answer 2

The problem you have is mostly caused by JNI you are using.

As your code wait during constructor for FileInputSream(String). That veryfie the existance of passed path and call a method private native void open(String).

Then openJDK implementation of FileInputSream#open(String) look like:

    JNIEXPORT void JNICALL
    Java_java_io_FileInputStream_open(JNIEnv *env, jobject this, jstring path) {
        fileOpen(env, this, path, fis_fd, O_RDONLY);
    }

This move us to io_util_md.c and method

jlong winFileHandleOpen(JNIEnv *env, jstring path, int flags)

You can analyse the code there.

---

At this point you have various options.

• Check the different JDK
• Write a C code, to creaete onw JNI method.
• Check the differnet file system.