Prevent OutOfMemory when using java.nio.MappedByteBuffer

Question

Consider application, which create 5-6 threads, each thread in cycle allocate MappedByteBuffer for 5mb page size.

MappedByteBuffer b = ch.map(FileChannel.MapMode.READ_ONLY, r, 1024*1024*5);

Sooner or later, when application works with big files, oom is thrown

java.io.IOException: Map failed  at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:758)
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:755)

According to specification, MappedBuffer should dispose direct memory as soon as it is GC itself. Looks like the problem is, that MappedBuffer-s are GC-ed too late, later then direct memory finished.

How to avoid this situation ? Probably say MappedBuffer to dispose implicitly or use some kind of pool of MappedBuffer

Answer 1

You can avoid having to trigger a GC by cleaning up the mapped byte buffers directly.

public static void clean(ByteBuffer bb) {
    if(bb == null) return;
    Cleaner cleaner = ((DirectBuffer) bb).cleaner();
    if(cleaner != null) cleaner.clean();
}

Provided you call this before discarding, you won't run out of virtual memory.

Perhaps you can look at creating larger ByteBuffers less often (unless you have a large number of files) Creating a MappedByteBuffer is not free (takes about 50 micro-seconds on some machines)