Write heavy, replicated, bigger-than-memory key-value store

Question

I'm looking for a key value store that can be used from an EC2 instance.

• item is just an unstructured string, no indexing required
• item size up to ~5MB but usually below 10kB
• lots of writes
• reading doesn't need to be fast, memcache can be put in front that caches frequently needed reads
• data is too big to fit into memory
• Eventual Consistency is fine
• daemon that can be accessed from multiple machines is required

Ideally something AWS hosted would be perfect but:

• S3 doesn't fit because of too many writes
• SimpleDB/DynamoDb don't fit because of item size limits and indexing is not required

As there are a lot of key value stores on the market it's hard to choose the best one. Which one would you recommend?

Answer 1

I found the perfect solution for my use case: memcachedb

It doesn't do fancy document/indexing, it's just a simple key value store.

I didn't do any performance testing yet though.

Edit:

We dropped memcachedb due to problems with replication. Instead we run now mongodb. Mongodb requires much more disk space, and more resources in general. But the replica sets work very reliable and are easy to set up.