Versioned Object/JSON Mapping to/from Mongo?

Question

So I have an app that needs to store certain configuration info, and so I am planning on storing the configs as simple JSON documents in Mongo:

appConfig: {
    fizz: true,
    buzz: 34
}

This might map to a Java POJO/entity like:

public class AppConfig {
    private boolean fizz;
    private int buzz;
}

etc. Ordinarily, with relational databases, I use Hibernate/JPA for O/R mapping from table data to/from Java entities. I believe the closest JSON/Mongo companion to table/Hibernate is a Morphia/GSON combo: use Morphia to drive connectivity from my Java app to Mongo, and then use GSON to O/J map the JSON to/from Java POJOs/entities.

The problem here is that, over time, my appConfig document structure will change. It may be something simple like:

appConfig: {
    fizz: true,
    buzz: 34
    foo: "Hello!"
}

Which would then require the POJO/entity to become:

public class AppConfig {
    private boolean fizz;
    private int buzz;
    private String foo;
}

But the problem is that I may have tens of thousands of JSON documents already stored in Mongo that don't have foo properties in them. In this specific case, the obvious solution is to set a default on the property like:

public class AppConfig {
    private boolean fizz;
    private int buzz;
    private String foo = "Hello!"
}

However in reality, eventually the AppConfig document/schema/structure might change so much that it in no way, shape or form resembles its original design. But the kicker is: I need to be backwards-compatible and, preferably, be capable of updating/transforming documents to match the new schema/structure where appropriate.

My question: how is this "versioned document" problem typically solved?

Answer 1

I usually solve this problem by adding a version field to each document in the collection.

You might have several documents in the AppConfig collection:

{
    _id: 1,
    fizz: true,
    buzz: 34
}

{
    _id: 2,
    version: 1,
    fizz: false,
    buzz: 36,
    foo: "Hello!"
}

{
    _id: 3,
    version: 1,
    fizz: true,
    buzz: 42,
    foo: "Goodbye"
}

In the above example, there are two documents at version one, and one older document at version zero (in this pattern, I generally interpret a missing or null version field to be version zero, because I always only add this once I'm versioning by documents in production).

The two principles of this pattern:

1. Documents are always saved at the newest version when they are actually modified.
2. When a document is read, if it's not at the newest version, it gets transparently upgraded to the newest version.

You do this by checking the version field, and performing a migration when the version isn't new enough:

DBObject update(DBObject document) {
    if (document.getInt("version", 0) < 1) {
        document.put("foo", "Hello!"); //add default value for foo
        document.put("version", 1);
    }
    return document;
}

This migration can fairly easily add fields with default values, rename fields, and remove fields. Since it's located in application code, you can do more complicated calculations as necessary.

Once the document has been migrated, you can run it through whatever ODM solution you like to convert it into Java objects. This solution no longer has to worry about versioning, since the documents it deals with are all current!

With Morphia this could be done using the @PreLoad annotation.

Two caveats:

1. Sometimes you may want to save the upgraded document back to the database immediately. The most common reasons for this are when the migration is expensive, the migration is non-deterministic or integrates with another database, or you're in a hurry to upgrade an old version.

2. Adding or renaming fields that are used as criteria in queries is a bit trickier. In practice, you may need to perform more than one query, and unify the results.

In my opinion, this pattern highlights one of the great advantages of MongoDB: since the documents are versioned in the application, you can seamlessly migrate data representations in the application without any offline "migration phase" like you would need with a SQL database.

Answer 2

The JSON deserialzer solves this in a very simple way for you, (using JAVA)

Just allow your POJO/entity to grown with new fields. When you deserialize your JSON from mongo to you entity - all missing fields will be null.

    mongoDocument v1    :        Entity of v3
    {
      fizz="abc",        -->       fizz      = "abc";
      buzz=123           -->       buzz      = 123;
                         -->       newObj    = null;
                         -->       obj_v3    = null;

   }

You can even use this the other way around if you like to have you legacy servers work with new database objects:

    mongoDocument v3     :        Entity of v1
    {
      fizz:"abc",        -->       fizz      = "abc";
      buzz:123,          -->       buzz      = 123;
      newObj:"zzz",      -->       
      obj_v3:"b          -->       

   }

Depending if they have the fields or not - it will be populated by the deserializer. Keep in mind that booleans are not best suited for this since they can default to false (depending on which deserializer you use).

So unless you are actively going to work with versioning of your objects why bother with the overhead when you can build a legacy safe server implementation what with just a few null checks can handle any of the older objects.

I hope this proposal might help you with your set-up