Equivalent of an ORM for a distributed key/value store?

Question

I'm in the process of evaluating how to implement something using a distributed key/value store for the back end. I'd like to have a layer on top of the key/value supporting an object model that is similar to what I'd get from an object-relational mapper.

Can anyone point me at any examples of other people doing this? I'm mostly looking for design ideas, though if I run across anything that I like enough I may just use it instead of writing my own. I'm probably going to wind up implementing mine in Perl on top of Riak, but those decisions are not final.

Answer 1

We have previously used Riak to do something similar, using the Ruby client Ripple which exposes an AciveModel interface. However, I do have to really advise against it (as others have). Using a heavy ORM on top of a key/value store you really do lose it's main advantage, which is speed.

We are now moving towards skipping Ripple and talking directly to Riak for a lot of speed conscious things (we are also moving to Erlang and using the PBC rather than HTTP interface, but that's another story :D), here is how we do it:

• In our objects we store a JSON document, in a Ripple compatible format. Although we have a requirement of this as we still use Ripple for some things, if I were to do this again without Ripple I would still probably use this format.

• Use Riak links to join objects together, don't store foreign keys in the document itself. Be advised there is a limit to the number of links you can store on an object, so don't go too crazy with them (e.g. storing a link to each comment on the user object).

• Ripple (and Riak) doesn't support indexes, so we had to roll our own solution. As an example we store a user object with a randomly generated key, 'fen2nf4j9fecd' in the 'users' bucket. We also store an object with the key 'tom' in the 'users_index_by_username' bucket with a Riak link to the object in the 'users' bucket. That way we can easily find which user has the username 'tom'.

You may also want to look into using key filtering. I haven't played with it yet, however I have seen performance figures that look quite good. You need to be careful with Riak not to list the keys of a bucket as due to the way it is implemented, Riak searches all keys, not just that bucket's keys.

Riak is quite a beast, however once you get your head around it you will love it. It make's replication effortless, and it does 'just work'.

Answer 2

You shouldn't really need much, if any, layer for this.

It's a key/value store for pete's sake, use whatever serialization mechanism exists in your language to convert to and from your typed object to the back-end's object. What else is there to do?

ORM's are far more complicated because they are dealing with a relational model on one side. A key value store, well, doesn't.