How are Google App Engine model classes stored?

Question

I'm in doubt how the objects are stored. Say I have a class defined like:

class SomeEntity(db.Model):
    some_number = db.IntegerProperty(required=True)

    def calculate_something(self):
        return self.some_number * 2

My guess is that the only thing stored in the data store is the name/value/type of some_number together with the fully qualified name of the class (SomeEntity). However I have not stumbled upon any information that confirms this.

1) Can anyone confirm this?

I would like to confirm that I can change (and add/remove) methods without somehow affecting the data is stored.

2) Furthermore, what happens to existing objects if I add a new property to the class (and what if that property has required=true)?

Answer 1

Entities are stored in the datastore in a protobuf representation (including its key - which includes your App ID and the entity's Kind). The Life of a Datastore Write article talks more about the representation of entities and how they are written to the datastore. Check out the rest of the articles in this series for more detailed information.

1) Methods have no bearing on the data stored with your entity, so you can add/remove/change these without affecting the representation of your data.

2) The datastore is schemaless (unlike the typical SQL database). Changing your Model has no impact on the data in the datastore at all. When you retrieve an existing entity, if it is missing a required field then an error will be raised. Alternatively, if you don't make it required and provide a default, then the default will be used for the missing field.

If you need to migrate an old model to a new one, you might want to consider using the appengine-mapreduce library to iterate over all of your entities and migrate each one individually. Read more about schema migration here.

Answer 2

They are stored as protocol buffers. You can read about some of the details in the "How Entities and Indexes are Stored" article.

You can see what is actually stored with:

db.model_to_protobuf(your_entity)

It is safe to add / remove methods, just be careful about overwriting built-in methods.

Include a default value if you add a property that is required. Existing entities will not be updated until you re-put the entity.