Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Difference between ByteBuffer.allocateDirect() and MappedByteBuffer.load()

I was trying to implement a sort of shared cache between two or more JVMs by memory mapping a particular file using MappedByteBuffer. From the specifications I see that when we use MappedByteBuffer.load() it should load the data into a direct buffer. I have a couple of questions on this.

My code snippet::

RandomAccessFile file = new RandomAccessFile("file.txt","rw");
FileChannel fc = file.getChannel();
MappedByteBuffer buf5 = fc.map(MapMode.READ_WRITE, 0, fc.size());

//ByteBuffer buf6 = ByteBuffer.allocateDirect(100000000);

buf5.load();

try
{
    Class c = Class.forName("java.nio.Bits");
    Field f = c.getDeclaredField("reservedMemory");
    f.setAccessible(true);
    long reservedMemory = f.getLong(null);
    f = c.getDeclaredField("maxMemory");
    f.setAccessible(true);
    System.out.println(
            "Direct Memory Usage: "+ reservedMemory +"/"+ f.getLong(null)+"\n");
}
catch (Throwable t)
{
}
  1. The output of the above code is 0 byte for the Direct Memory Usage (File.txt is 1 GB). But if I uncomment the line ..

    ByteBuffer buf6 = ByteBuffer.allocateDirect(100000000);
    

    I get a Direct Memory Usage of 100MB . Not able to understand why is this so, as to why I am not getting any direct memory usage in the first place ( i.e. when the line is commented out )

  2. Although the Direct Memory Usage is 0 B for the above code, I do see that the resident memory ( using unix top ) of the process increase by 1 GB. But if I do a "free -m" on the box, I do not see any increase in memory usage.

In both the cases, I am a bit confused as to where the memory is ending up.

Thanks!

like image 684
sim Avatar asked Aug 04 '09 18:08

sim


People also ask

What is a direct ByteBuffer?

A direct buffer is a chunk of native memory shared with Java from which you can perform a direct read. An instance of DirectByteBuffer can be created using the ByteBuffer.

What is ByteBuffer in Java?

ByteBuffer holds a sequence of integer values to be used in an I/O operation. The ByteBuffer class provides the following four categories of operations upon long buffers: Absolute and relative get method that read single bytes. Absolute and relative put methods that write single bytes.

What does ByteBuffer wrap do?

wrap. Wraps a byte array into a buffer. The new buffer will be backed by the given byte array; that is, modifications to the buffer will cause the array to be modified and vice versa. The new buffer's capacity and limit will be array.

What is the byte order of ByteBuffer?

By default, the order of a ByteBuffer object is BIG_ENDIAN. If a byte order is passed as a parameter to the order method, it modifies the byte order of the buffer and returns the buffer itself. The new byte order may be either LITTLE_ENDIAN or BIG_ENDIAN.


1 Answers

Direct ByteBuffers (those allocated using ByteBuffer.allocateDirect) are different to MappedByteBuffers in that they represent different sections of memory and are allocated differently. Direct ByteBuffers are a way to access a block of memory allocated outside of the JVM generally allocated with a malloc call (although most implementations will probably use an efficient slab allocator). I.e. it's just a pointer to a block of memory.

A MappedByteBuffer represents a section of memory allocated using mmap call, which is used to perform memory mapped I/O. Therefore MappedByteBuffers won't register their use of memory in the same way a Direct ByteBuffer will.

So while both are "direct" in that they represent memory outside of the JVM their purposes are different.

As an aside, in order to get the reservedMemory value you are reflectively calling to an internal method of the JVM, whose implementation is not covered by any specification, therefore there are no guarantees as to what that value returns. Direct ByteBuffers can be allocated from within JNI using NewDirectByteBuffer call from C/C++ (MappedByteBuffers likely use this) and this probably doesn't affect the reservedMemory value, which is may only changed when using the Java ByteBuffer.allocateDirect.

like image 88
Michael Barker Avatar answered Sep 20 '22 22:09

Michael Barker