Memory-mapped files in Java

Question

I've been trying to write some very fast Java code that has to do a lot of I/O. I'm using a memory mapped file that returns a ByteBuffer:

public static ByteBuffer byteBufferForFile(String fname){
    FileChannel vectorChannel;
    ByteBuffer vector;
    try {
        vectorChannel = new FileInputStream(fname).getChannel();
    } catch (FileNotFoundException e1) {
        e1.printStackTrace();
        return null;
    }
    try {
        vector = vectorChannel.map(MapMode.READ_ONLY,0,vectorChannel.size());
    } catch (IOException e) {
        e.printStackTrace();
        return null;
    }
    return vector;
}

The problem that I'm having is that the ByteBuffer .array() method (which should return a byte[] array) doesn't work for read-only files. I want to write my code so that it will work with both memory buffers constructed in memory and buffers read from the disk. But I don't want to wrap all of my buffers a ByteBuffer.wrap() function because I'm worried that this will slow things down. So I've been writing two versions of everything, one that takes a byte[], the other that takes a ByteBuffer.

Should I just wrap everything? Or should I double-write everything?

Answer 1

Did anyone actually check to see if ByteBuffers created by memory mapping support invoking .array() in the first place, regardless of readonly/readwrite?

From my poking around, as far as I can tell, the answer is NO. A ByteBuffer's ability to return a direct byte[] array via ByteBuffer.array() is goverened by the presence of ByteBuffer.hb (byte[]), which is always set to null when a MappedByteBuffer is created.

Which kinda sucks for me, because I was hoping to do something similar to what the question author wanted to do.

Answer 2

Its always good not to reinvent the wheels. Apache has provided a beautiful library for performing I/O operations. Take a look at http://commons.apache.org/io/description.html

Here's the scenario it serves. Suppose you have some data that you'd prefer to keep in memory, but you don't know ahead of time how much data there is going to be. If there's too much, you want to write it to disk instead of hogging memory, but you don't want to write to disk until you need to, because disk is slow and is a resource that needs tracking for cleanup.

So you create a temporary buffer and start writing to that. If / when you reach the threshold for what you want to keep in memory, you'll need to create a file, write out what's in the buffer to that file, and write all subsequent data to the file instead of the buffer.

That's what DeferredOutputStream does for you. It hides all the messing around at the point of switch-over. All you need to do is create the deferred stream in the first place, configure the threshold, and then just write away to your heart's content.

EDIT: I just did a small re-search using google and found this link: http://lists.apple.com/archives/java-dev/2004/Apr/msg00086.html (Lightning fast file read/write). Very impressive.