Django REST Errors when serializing model with ManyToManyField

Question

I've created a class modeling a group of files within a software product build, on a Django server using the Django-REST package. The design is that the group of files (a Depot instance) should be able to be assigned to multiple Build instances (e.g. both the "alpha" and "beta" builds using the same exact audio file depot). However, at the time that the depot is created, it is being created as part of the creation of single Build on the client; it is only later that a utility script will allow an existing Depot to be added to other Builds.

It seemed natural to me that the Depot class should represent this relationship with a ManyToManyField. The problem is that the serializer does not seem to know what to do with this ManyToManyField. I've tried several workarounds, but each has its own error. I've tried having my DepotSerializer be either a rest_framework.serializers.Serializer or a rest_framework.serializers.ModelSerializer, but that seems largely unrelated to this problem.

Models.py:

class Depot(models.Model):
    name = models.CharField(max_length=64)
    builds = models.ManyToManyField(Build)

    TYPE_EXECUTABLE = 0
    TYPE_CORE = 1
    TYPE_STREAMING = 2
    depot_type = models.IntegerField(choices = (
        (TYPE_EXECUTABLE, 'Executable'),
        (TYPE_CORE, 'Core'),
        (TYPE_STREAMING, 'Streaming'),
    ))

    def __str__(self):
        return self.name

Views.py:

class DepotCreate(mixins.CreateModelMixin,
                  generics.GenericAPIView):
    serializer_class = DepotSerializer
    queryset = Depot.objects.all()

    def post(self, request, *args, **kwargs):
        return self.create(request, *args, **kwargs)

Serializers.py version 1:

class DepotSerializer(serializers.ModelSerializer):
    builds = serializers.PrimaryKeyRelatedField()

    class Meta:
        model = Depot
        fields = ('id', 'name', 'builds', 'depot_type')
        read_only_fields = ('id',)

    def validate(self, attrs):
        build = attrs['builds']
        if build == None:
            raise serializers.ValidationError("Build could not be found")

        for depot in build.depot_set.all():
            if depot.name == attrs['name']:
                raise serializers.ValidationError("Build already contains a depot \"{}\"".format(depot.name))

        return attrs

    def restore_object(self, attrs, instance=None):
        # existence of the build has already been validated
        return Depot(**attrs)

This version results in the following error during the Depot init call:

Exception Type: TypeError
Exception Value:    
'builds' is an invalid keyword argument for this function
Exception Location: /webapps/cdp_admin_django/lib/python3.4/site-packages/django/db/models/base.py in __init__, line 417

That appears to indicate that the Depot model cannot handle the 'builds' parameter despite the fact that it has a 'builds' ManyToManyField member.

Serializers.py 'restore_object' ver 2:

def restore_object(self, attrs, instance=None):
    # existence of the build has already been validated
    build = attrs['builds']

    depotObj = Depot(name=attrs['name'], depot_type=attrs['depot_type'])
    depotObj.builds.add(build)
    return depotObj

This gave me the error:

Exception Type: ValueError
Exception Value:    
"<Depot: depot_test4>" needs to have a value for field "depot" before this many-to-many relationship can be used.
Exception Location: /webapps/cdp_admin_django/lib/python3.4/site-packages/django/db/models/fields/related.py in __init__, line 524

After quite a bit of investigation, I found that ManyToMany relationships can give you trouble if you don't save the MYSQL entry before attempting to manipulate that field. Hence, restore_object ver 3:

def restore_object(self, attrs, instance=None):
    # existence of the build has already been validated
    build = attrs['builds']

    depotObj = Depot(name=attrs['name'], depot_type=attrs['depot_type'])
    depotObj.save()
    depotObj.builds.add(build)
    return depotObj

This does successfully create the table entry for this instance, but ends up throwing the following error:

Exception Type: IntegrityError
Exception Value:    
(1062, "Duplicate entry '5' for key 'PRIMARY'")
Exception Location: /webapps/cdp_admin_django/lib/python3.4/site-packages/MySQLdb/connections.py in defaulterrorhandler, line 38

This error takes place during rest_framework/mixins.py call to serializer.save(force_insert=True). Which looks like it is supposed to force the creation of a new table entry, presumably disagreeing with my earlier call to Model.save.

Does anyone know the correct approach for a design like this? I feel like this can't be that unusual of a table structure.

EDIT 10/20/2014: After the suggestion below, I experimented with writing a new ModelSerializer for one of my models; for the most part because of these types of order-of-operations problems, I'd backed off from using ModelSerializer and did all of my data-to-object field processing in views.py by reading serializer.data.

Having a PrimaryKeyRelatedField(many=True) in the ModelSerializer DID help. Notably, I was able to create a serializer instance with existent models and get the correct serializer.data. However, I still have the problem where restore_object can do everything except create a new model instance and pass down the ManyToManyField value. I still get "TypeError: '[PrimaryKeyRelatedField name]' is an invalid keyword argument for this function" if I pass the field to the model's init func. I still cannot save the model before the REST library does it itself. In addition, in this mode, the serializer populates serializer.data with the values of the Model, not the values provided in the data input. So if you do not use the PrimaryKeyRelatedField's attrs value in restore_object, it is discarded.

It appears that I need to override ModelSerializer.save to some kind of a pre-save, apply ManyToMany input, and a post-save, but I would need the attrs values so I can apply and modify the ManyToManyField at that time. I realize that the serializer does have the init_data field to see the original inputs, but in the case where the serializer is being used to deserialize a list of data into a list of new objects, I don't think there's a way to trace which serializer.init_data corresponds with which serializer.object.

Answer 1

In your serializer version 1, you do not have to add

builds = serializers.PrimaryKeyRelatedField()

as the model serializer will create this for you. In fact if you look at the exemple of the documentation (http://www.django-rest-framework.org/api-guide/relations/) you'll see that the PrimaryKeyRelatedField is applied when there is a FK 'to' the current model (not a M2M relation).

I would remove this from the serializer and see then what's going on.