Following backreferences of unknown kinds in NDB

Question

I'm in the process of writing my first RESTful web service atop GAE and the Python 2.7 runtime; I've started out using Guido's shiny new ndb API.

However, I'm unsure how to solve a particular case without the implicit back-reference feature of the original db API. If the user-agent requests a particular resource and those resources 1 degree removed:

host/api/kind/id?depth=2

What's the best way to discover a related collection of entities from the "one" in a one-to-many relationship, given that the kind of the related entity is unknown at development time?

• I'm unable to use a replacement query as described in a previous SO inquiry due to the latter restriction. The fact that my model is definable at runtime (and therefore isn't hardcoded) prevents me from using a query to filter properties for matching keys.

• Ancestor and other kindless queries are also out due to the datastore limitation that prevents me from filtering on a property without the kind specified.

Thus far, the only idea I've had (beyond reverting to the db api) is to use a cross-group transaction to write my own reference on the "one", either by updating an ndb.StringProperty(repeat=True) containing all the related kinds when an entity of a new kind is introduced or by simply maintaining a list of keys on the "one" ndb.KeyProperty(repeat=True) every time a related "many" entity is written to the datastore.

I'm hoping someone more experienced than myself can suggest a better approach.

Given jmort253's suggestion, I'll try to augment my question with a concrete example adapted from the docs:

class Contact(ndb.Expando):
    """ The One """

    # basic info
    name = ndb.StringProperty()
    birth_day = ndb.DateProperty()

    # If I were using db, a collection called 'phone_numbers' would be implicitly 
    # created here.  I could use this property to retrieve related phone numbers 
    # when this entity was queried.  Since NDB lacks this feature, the service 
    # will neither have a reference to query nor the means to know the 
    # relationship exists in the first place since it cannot be hard-coded.  The
    # data model is extensible and user-defined at runtime; most relationships
    # will be described only in the data, and must be discoverable by the server.
    # In this case, when Contact is queried, I need a way to retrieve the
    # collection of phone numbers.

    # Company info.
    company_title = ndb.StringProperty()
    company_name = ndb.StringProperty()
    company_description = ndb.StringProperty()
    company_address = ndb.PostalAddressProperty()

class PhoneNumber(ndb.Expando):
    """ The Many """

    # no collection_name='phone_numbers' equivalent exists for the key property
    contact = ndb.KeyProperty(kind='Contact')
    number = ndb.PhoneNumberProperty()

Answer 1

Interesting question! So basically you want to look at the Contact class and find out if there is some other model class that has a KeyProperty referencing it; in this example PhoneNumber (but there could be many).

I think the solution is to ask your users to explicitly add this link when the PhoneNumber class is created.

You can make this easy for your users by giving them a subclass of KeyProperty that takes care of this; e.g.

class LinkedKeyProperty(ndb.KeyProperty):
    def _fix_up(self, cls, code_name):
        super(LinkedKeyProperty, self)._fix_up(cls, code_name)
        modelclass = ndb.Model._kind_map[self._kind]
        collection_name = '%s_ref_%s_to_%s' % (cls.__name__,
                                               code_name,
                                               modelclass.__name__)
        setattr(modelclass, collection_name, (cls, self))

Exactly how you pick the name for the collection and the value to store there is up to you; just put something there that makes it easy for you to follow the link back. The example would create a new attribute on Contact:

Contact.PhoneNumber_ref_contact_to_Contact == (PhoneNumber, PhoneNumber.contact)

[edited to make the code working and to add an example. :-) ]

Answer 2

Sound like a good use case for ndb.StructuredProperty.