creating mongoengine class dynamicly from json schema

Question

i'm looking for a way to take my json schema and dynamically create a mongoengine class at runtime.

for example: the mongoengine class written below

class user(Document):
     _id  = StringField(required=False) # mongodb id
     name = StringField(required=True)  # user name
     email= StringField(required=False,regex="^[a-zA-Z0-9]*@mydomain.com$")  # user email

will be the same as the dynamically generated at runtime class generated from this schema

{
"type":"object",
"properties":{
    "_id"      : {"type":"string"},
    "name"     : {"type":"string"},
    "email"    : {"pattern":"^[a-zA-Z0-9]*@mydomain.com$"}
    }
}

any suggestions?

Answer 1

You can create python classes dynamically, like below:

user_properties = { 
   '_id': StringField(required=False), # mongodb id
   'name': StringField(required=True), # user name
   'email': StringField(required=False,regex="^[a-zA-Z0-9]*@mydomain.com$"),  # user email
}
User = type("User", (Document, ), user_properties)

The remaining is only convertor from dict with "your schema" provided on input, that will be converted to dict user_properties .

Also, here is my answer to similar question "How to generate a model for a mongoengine Document from an already existing collection" which can also help.

Answer 2

The jinja template engine has features that can be utilized to create a mongo class template that populates the property schemas in the class definition based on the type of each property. Then, we generate the desired Python class dynamically using a simple intermediate script that renders this template. The following is an example illustrating this concept. We start with the following project structure, which has a sample parent folder called Templates and three files:

Templates\
    - mongo_sample.schema.json
    - mongoclass_template.jinja
    - auto_mongo_class.py

Let's explore these files:

1. mongo_sample.schema.json:

This is the MongoDB schema document. A consistent schema helps create a standard Jinja template later:

{
  "type":"object",
  "required": ["name"],
  "properties":{
      "_id"      : {"type":"string"},
      "name"     : {"type":"string"},
      "email"    : {"pattern":"^[a-zA-Z0-9]*@mydomain.com$"}
    }
}

2. mongoclass_template.jinja:

This is the Jinja template for a Mongo class which will be used to render the desired Python Mongo class in the end. It makes use of the Jinja template language. Here's the reference for the Template Designer Documentation. Here, all we are doing is creating a template that covers all "type" cases and also parameters like "required" and "regex". Notice that a consistent schema is important:

from mongoengine import DynamicDocument, StringField, IntField

class Student(DynamicDocument):       
    {%- for prop, val in properties.items() %} 
    {%- if prop in required -%}
    {% set req = True %}
    {% else %}    
    {% set req = False %}
    {%- endif -%}
    {%- if val.pattern is defined -%}
    {% set patt = val.pattern %}    
    {% else %}
    {% set patt = None %} 
    {%- endif -%}  
    {%- if val.type == "string" -%}
    {{prop}} = StringField(required={{req}}, regex={{patt}})
    {%- elif val.type == "int" -%}
    {{prop}} = IntField(required={{req}})        
    {%- else -%} 
    {{prop}} = StringField(required={{req}}, regex="{{patt}}")       
    {%- endif -%}
    {%- endfor %}
    

3. auto_mongo_class.py:

This Python script parses through the JSON schema above to get the properties object, passes the necessary variables to the template render() method which will be used while rendering the template and finally writes to a Python file called models.py which is the final Mongo class:

import json
from jinja2 import Template

with open(r"mongo_sample.schema.json", "r") as schema_json:
    schema_tree = json.load(schema_json)
properties = schema_tree["properties"]
required = schema_tree["required"]
template = Template(open(r"mongoclass_template.jinja").read()).render(properties=properties, \
    required=required)

with open("models.py", 'w') as file_:
    file_.write(template)

Now, let's run this script:

python auto_mongo_class.py

A new Python script called models.py will appear in the project folder with the following contents:

from mongoengine import DynamicDocument, StringField, IntField

class Student(DynamicDocument):    
    
    _id = StringField(required=False, regex=None)
    
    name = StringField(required=True, regex=None)    
        
    email = StringField(required=False, regex="^[a-zA-Z0-9]*@mydomain.com$")