Does a good framework for MongoDB "schema" upgrades in Scala exist?

Question

What are the options for MongoDB schema migrations/upgrades?

We (my colleagues and I) have a somewhat large (~100 million record) MongoDB collection. This collection is mapped (ORM'd) to a Scala lift-mongodb object that has been through a number of different iterations. We've got all sorts of code in there which handles missing fields, renames, removals, migrations, etc.

As much as the whole "schema-less" thing can be nice and flexible, in this case it's causing a lot of code clutter as our object continues to evolve. Continuing down this "flexible object" path is simply not sustainable.

How have you guys implemented schema migrations/upgrades in MongoDB with Scala? Does a framework for this exist? I know that Foursquare uses Scala with MongoDB and Rogue (their own query DSL)... does anyone know how they handle their migrations?

Thank you.

Answer 1

Perhaps this can help somewhat, this is how Guardian.co.uk handle this:

http://qconlondon.com/dl/qcon-london-2011/slides/MatthewWall_WhyIChoseMongoDBForGuardianCoUk.pdf

Schema upgrades

This can be mitigated by:

• Adding a “version” key to each document
• Updating the version each time the application modifies a document
• Using MapReduce capability to forcibly migrate documents from older versions if required

Answer 2

I program migrations of MongoDB data with my own Scala framework "Subset". It lets define document fields pretty easily, fine tune data serialization (e.g. write "date" is a specific format and so on) and build queries and update modifiers in terms of the defined fields. This "gist" gives a good introduction