List of References in Google App Engine for Python

Question

In Google App Engine, there is such a thing as a ListProperty that allows you to hold a list (array) of items. You may also specify the type of the item being held, for instance string, integer, or whatever.

Google App Engine also allows you to have a ReferenceProperty. A ReferenceProperty "contains" a reference to another Google App Engine Model entity. If you access a ReferenceProperty, it will automatically retrieve the actual entity that the reference points to. This is convenient, as it beats getting the key, and then getting the entity for said key.

However, I do not see any such thing as a ListReferenceProperty (or ReferenceListProperty). I would like to hold a list of references to other entities, that would automatically be resolved when I attempt to access elements within the list. The closest I can get it seems is to hold a list of db.Key objects. I can use these keys to then manually retrieve their associated entities from the server.

Is there any good solution to this? Basically, I would like the ability to have a collection of (auto-dereferencing) references to other entities. I can almost get there by having a collection of keys to other entities, but I would like it to "know" that these are key items, and that it could dereference them as a service to me.

Thank you

Answer 1

Step one:

Use db.ListProperty(db.Key) to create the relationship. You want the ListProp to be on the Entity that will have the fewer references in the Many to Many relationship. This will also give you a back reference. So:

class Spam
  prop1 = db.String
  eggs = db.List

class Eggs
  prop1 = db.string
  @property
  def spams(self):
    return Spam.all().filter('eggs', self.key())

This provides a References both ways.

Step two:

Create a utlility method that derefrences properties.

def prefetch_refprops(entities, *props):
    """Dereference Reference Properties to reduce Gets.  See:
    http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine
    """
    fields = [(entity, prop) for entity in entities for prop in props]
    ref_keys = [prop.get_value_for_datastore(x) for x, prop in fields]
    ref_entities = dict((x.key(), x) for x in db.get(set(ref_keys)))
    for (entity, prop), ref_key in zip(fields, ref_keys):
        prop.__set__(entity, ref_entities[ref_key])
    return entities  

Usage would be:

derefrenced_spams = prefetch_refprops(Spams, models.Spam.eggs)