Loading db.ListProperty() with AppEngine bulkloader

Question

I'm trying to populate a db.ListProperty() model field using bulkloader.

I'm using an import transform function as follows:

def parse_array(fn):
    def wrapper(value):
        return [fn(seg) for seg in re.split("\\,", value) if not seg=='']
   return wrapper

Configured as follows:

import_transform: lib.bulkloader_helpers.parse_array(int)

Some of the arrays are empty, and these are causing a problem. When I upload:

BadValueError: May not use the empty list as a property value; property xxx is [].

Uh oh. Okay, let's change the import transform function:

def parse_array(fn):
    def wrapper(value):
        args[fn(seg) for seg in re.split("\\,", value) if not seg=='']
        if args==[]:
            return None
        else:
            return args 
    return wrapper

Now the empty lists load just fine. However when the app tries to load the model:

BadValueError: Property xxx is required

Nor can I set db.ListProperty(required=False):

google.appengine.ext.db.ConfigurationError: required must be True.

Anyone suggest a way out ?

Thanks,

Justin

Answer 1

This is a known bug (Issue 3646) with the App Engine development environment.

A workaround is to change (as of version 1.9.6) line 1530 of google.appengine.api.datastore_types.py which reads:

  if not values:

To:

  if not values_type is list and not values:

Afterwards you can properly insert

[]

for the empty arrays and it should work.