Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Key/Value database for storing binary data

I am looking for a lightweight, reliable and fast key/value database for storing binary data. Simple without server. Most of popular key/value databases like CDB and BerkeleyDB does not natively store BLOB. What can be the best choice that I missed?

My current choice is SQLite, but it is too advanced for my simple usage.

like image 287
Googlebot Avatar asked Mar 21 '12 11:03

Googlebot


People also ask

How do you store binary data?

Binary data can be stored in a table using the data type bytea or by using the Large Object feature which stores the binary data in a separate table in a special format and refers to that table by storing a value of type oid in your table.

What kinds of data can you store in a key-value database?

A key-value database stores data as a collection of key-value pairs in which a key serves as a unique identifier. Both keys and values can be anything, ranging from simple objects to complex compound objects.

When should you not use a key-value database?

Key-value stores are not considered suitable for applications requiring frequent updates or for complex queries involving specific data values, or multiple unique keys and relationships between them.

Which is the best example for key-value store?

A telephone directory is a good example, where the key is the person or business name, and the value is the phone number. Stock trading data is another example of a key-value pair.


2 Answers

As it was previously pointed out, BerkeleyDB does support opaque values and keys, but I would suggest a better alternative: LevelDB.

LevelDB:

Google is your friend :), so much so that they even provide you with an embedded database: A fast and lightweight key/value database library by Google.

Features:

  • Keys and values are arbitrary byte arrays.
  • Data is stored sorted by key.
  • Callers can provide a custom comparison function to override the sort order.
  • The basic operations are Put(key,value), Get(key), Delete(key).
  • Multiple changes can be made in one atomic batch.
  • Users can create a transient snapshot to get a consistent view of data.
  • Forward and backward iteration is supported over the data.
  • Data is automatically compressed using the Snappy compression library.
  • External activity (file system operations etc.) is relayed through a virtual interface so users can customize the operating system interactions.
  • Detailed documentation about how to use the library is included with the source code.
like image 83
Kiril Avatar answered Sep 30 '22 07:09

Kiril


What makes you think BerkDB cannot store binary data? From their docs:

Key and content arguments are objects described by the datum typedef. A datum specifies a string of dsize bytes pointed to by dptr. Arbitrary binary data, as well as normal text strings, are allowed.

Also see their examples:

money = 122.45;
key.data = &money;
key.size = sizeof(float);
...
ret = my_database->put(my_database, NULL, &key, &data, DB_NOOVERWRITE);
like image 45
hroptatyr Avatar answered Sep 30 '22 07:09

hroptatyr