Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Java disk-based key-value storage

Is there an efficient Java implementation of a filesystem-based key-value storage with the following features:

  1. Store, overwrite, and retrieve byte arrays by a unique ID (may be assigned by the storage)
  2. No memory caching (read means read from file system, write means write to file system immediately)
  3. Total data size up to few terabytes
  4. Number of stored objects up to hundreds of millions
  5. Manageable number of file system objects (to move/copy/delete entire storage on file system level)

Will Berkeley DB JE do?

like image 877
Andrey Logvinov Avatar asked Dec 14 '11 18:12

Andrey Logvinov


2 Answers

Simply format a dedicated partition with a file system of your choice? The file system would meet requirements 1-4, and requirement 5 can be met by moving/copying or deleting that partition.

like image 51
meriton Avatar answered Oct 28 '22 16:10

meriton


I suggest MapDB, MapDB provides concurrent Maps, Sets and Queues backed by disk storage or off-heap memory. Lightweight and hackable.

like image 20
Tai Zhou Avatar answered Oct 28 '22 15:10

Tai Zhou