need Java library -- A Local File Cache for Remote (eg S3) Files

Question

My application requires a layer between Java and the filesystem to make transparent the fact that the filesystem only contains a subset of all the files (which are stored on S3). The layer has to do a lot of what normal file IO does, which is to open files, lock them for reading/writing, etc., but when opening it has to possibly download files and evict closed ones. Another feature I need is that if a file is locked for reading/writing, an open call can unlock the file and close the existing stream (ie, kick the other user off). Another is management of temporary files.

Is there anything remotely similar that is open source, or do I just have to roll up my sleeves? Should I start from scratch, or are there some hooks in java IO that I should tap?

Answer 1

I would suggest you check apache commons vfs Even if it's not exactly what you need, you may find useful ideas from it.