Best database for storing statistics from Erlang

Question

I need to choose a database for storing statistical data (in fact this is a series of timestamp-value data). I understand that virtually any database can handle this, but there are a couple of requirements:

• it should be fast;
• it should be able to handle A LOT of data (10s of gigabytes) and splice it fast;
• it should have a stable, maintained and handy interface to Erlang;
• it should be available from Python;
• it should be able to make something like the thing named "capped collections" in mongodb: collection with the capped size, with old data being rewritten if the size reach the limit.

I thought about mongo, but emongo seems to be a little dead - the last commit was made 7 months ago.

Answer 1

Riak may be a good choice (here's a Riak comparison to MongoDB). It's written in Erlang, is distributed, fault tolerant and scales linearly. It has clients for Erlang, Javascript, Java, PHP, Python, Ruby. A REST interface, a protobuf interface and so many other goodies (Map Reduce, links, replication, pre/post commit hooks, ...). It's open source and is created maintained by Basho. Basho has commercial offering of Riak as well with some extra features (like multi-site replication, SNMP monitoring, etc) but there's awsome value in the OS version.

Depending on your needs it may make sense to combine a couple of technologies. For example you could front your system with an in memory store like Redis for speed and use Riak to persist the data. Redis + Riak is a pretty sweet stack.

Answer 2

I think postgresql and pgsql driver it will be best solution for you.

Answer 3

Files on disk, rotated, will serve your demands fine. The point is you don't want to search data quickly.