Create a model from YAML/JSON on the fly

Question

I am trying out MongoEngine, a DRM library for Python to use with MongoDB. I can define a model like this example from the site:

class User(Document):
    email = StringField(required=True)
    first_name = StringField(max_length=50)
    last_name = StringField(max_length=50)

It works like a charm and MongoEngine is really nice. Well, I want to go further and wonder if I can define my models somewhere in a file in JSON or YAML, or whatever so that then create a model using it. So this declaration could look like this in JSON:

{
    "model":"User",
    "fields":{
        "email":{
            "type":"string",
            "required":"true"
        },
        "first_name":{
            "type":"string",
            "max_length":"50"
        },
        "last_name":{
            "type":"string",
            "max_length":"50"
        }
    }
}

Then I would parse this JSON and create a model using it. May be it could be just a single import operation that I would perform each time I modify the model definition, or may be it could parse the whole JSON every time. Is that a good scenario? I just want to let people who will use the app define their own models without having to dig into the code. Any ideas how to do dynamic model creation are appreciated

Answer 1

If you're going to use YAML, pyyaml is completely painless, and automatically outputs a data structure using python's built-in types (or even more complex types that you define).

Any way you go, I would also highly recommend Rx as a validator so you can easily verify the integrity of the loaded files.*

As for using this to create a model you can use the built-in function type (not type(object), but type(name, bases, dict)) which... "[r]eturn a new type object. This is essentially a dynamic form of the class statement."

So, you can call:

def massage(fields_dict):
    #transform your file format into a valid set of fields, and return it

user_class = type(yaml_data['model'], Document, massage(yaml_data['fields']) )

*I've used both of these together in the last eight hours, coincidentally - they work together painlessly, e.g.:

import yaml
import Rx

data = yaml.load(open("foo.yaml")
rx = Rx.Factory({ "register_core_types": True })
schema = rx.make_schema(yaml.load(open("schema.yaml")))

if not schema.check(data):
    raise ValueError("data file contents are not in a valid format")