Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

java.io.RandomAccessFile scalability (or other options)

I'm working on some Java code that will eventually be used within an app server to access some really big files (over 1GB, under 20GB), possibly hosted on an NFS share. Servicing an individual request will involve doing this:

  1. Find the large file I need to read
  2. Navigate to a random point in that file
  3. Read bytes from that file (usually under 1MB)
  4. Return those bytes

I have some happy simple POC code at the moment that simply opens a new read-only file and the closes it:

RandomAccessFile raf=new RandomAccessFile(myFileName, "r");
try{
   byte[] buffer = new byte[size];
   raf.seek(position);
   raf.reafFully(buffer);
   return buffer;
}
finally{
   raf.close();
}

I'm wondering if this is an elegantly simple approach that should work really well, or a foolishly simplistic approach which will have a lot of problems under heavy load (and perhaps I need to make a thread-safe pool of readers, etc). Obviously testing that assumption would be best, but I was wondering if there were any best practices or known issues with either approach. So far I haven't been able to figure out very much googling...

Thanks!

PS. It's not clear yet whether the final version of this would be hosted on Windows or *nix. It's also not clear how the large files would be shared. PPS. The app servers are likely to be configured in a cluster, so two different app servers might need to read the same large shared file at the same time.

like image 593
Dave Avatar asked Nov 04 '22 10:11

Dave


1 Answers

Another option is java NIO, namely FileChannel. FileChannel is also navigable and it may be faster than RandomAccessFile because it can work with so called direct buffers. It's got some more interesting features, eg it is interruptible.

like image 141
Evgeniy Dorofeev Avatar answered Nov 09 '22 12:11

Evgeniy Dorofeev