Getting non_field_errors using JSONField()

Question

I'm trying to make a PATCH request using to the Django Rest Framework but get the following error:

{"image_data": [{"non_field_errors": ["Invalid data"]}]

I understand that JSONField() could give some issues so I have taken care of that by adding to_native and from_native, But, I'm still running into this issue. I don't think JSONField() is the problem here at all, but still worth mentioning.

I believe I'm doing something fundamentally wrong in how I am trying to update the related field.

Code below...

Models:

class Photo(models.Model):
    user = models.ForeignKey(AppUser, help_text="Item belongs to.")
    image_data = models.ForeignKey("PhotoData", null=True, blank=True)


class PhotoData(models.Model):
    thisdata = JSONField()

Serializers:

class ExternalJSONField(serializers.WritableField):
    def to_native(self, obj):
        return json.dumps(obj)

    def from_native(self, value):
        try:
            val = json.loads(value)
        except TypeError:
            raise serializers.ValidationError(
                "Could not load json <{}>".format(value)
            )
        return val

class PhotoDataSerializer(serializers.ModelSerializer):

    thisdata = ExternalJSONField()
    class Meta:
        model = PhotoData
        fields = ("id", "thisdata")


class PhotoSerializer(serializers.ModelSerializer):

    image_data = PhotoDataSerializer()

    class Meta:
        model = Photo
        fields = ("id","user", "image_data")

PATCH:

> payload = {"image_data": {"thisdata": "{}"}}
> requests.patch("/photo/123/",payload )

I have also tried:

> payload = {"image_data": [{"thisdata": "{}"}]}
> requests.patch("/photo/123/",payload )

But again giving the same error:

[{"non_field_errors": ["Invalid data"]}]

Answer 1

The original idea of Django Rest Framework's serialization of relations is to not change values of related fields. It means that your payload should contain a pk of PhotoData object, not a dataset for it. It's like in models you can't assign a dict to a foreign key field.

Good (works only with serializers.PrimaryKeyRelatedField which contains problems itself):

   payload = {"image_data": 2}

Bad (not works in DRF by default):

   payload = {"image_data": {'thisdata': '{}'}}

Actually the data model that you provided doesn't need PhotoData at all (you can move thisdata field to Photo), but let's assume you have a special case even when Zen of Python says Special cases aren't special enough to break the rules..

So, here is some possible ways:

Using fields serializers (your original way)

What you want to do now is possible but is very ugly solution. You may create a PhotoDataField (works for me, but not ready to use code, only for demonstration)

class PhotoDataField(serializers.PrimaryKeyRelatedField):

    def field_to_native(self, *args):
        """
        Use field_to_native from RelatedField for correct `to_native` result
        """
        return super(serializers.RelatedField, self).field_to_native(*args)

    # Prepare value to output
    def to_native(self, obj):
        if isinstance(obj, PhotoData):
            return obj.thisdata
        return super(PhotoDataField, self).to_native(obj)

    # Handle input value
    def field_from_native(self, data, files, field_name, into):
        try:
            int(data['image_data'])
        except ValueError:
            # Looks like we have a data for `thisdata` field here.
            # So let's do write this to PhotoData model right now.
            # Why? Because you can't do anything with `image_data` in further.
            if not self.root.object.image_data:
                # Create a new `PhotoData` instance and use it.
                self.root.object.image_data = PhotoData.objects.create()
            self.root.object.image_data.thisdata = data['image_data']
            self.root.object.image_data.save()

            return data['image_data']
        except KeyError:
            pass
        # So native behaviour works (e.g. via web GUI)
        return super(PhotoDataField, self).field_from_native(data, files, field_name, into)

and use it in PhotoSerializer

class PhotoSerializer(serializers.ModelSerializer):

    image_data = PhotoDataField(read_only=False, source='image_data')

    class Meta:
        model = Photo
        fields = ("id", "user", "image_data")

so the request will works well

payload = {"image_data": '{}'}
resp = requests.patch(request.build_absolute_uri("/api/photo/1/"), payload)

and the "good" request also

photodata = PhotoData.objects.get(pk=1)
payload = {"image_data": photodata.pk}
resp = requests.patch(request.build_absolute_uri("/api/photo/1/"), payload)

and in result you will see in GET request "image_data": <photodata's thisdata value>,.

But, even if you will fix the validation problems with this approach it still be a huge pain in ass as you can see from my code (this is only thing DRF can offers you when you want to "break a normal workflow", Tastypie offers more).

Normalize your code and use @action (recommended)

class PhotoDataSerializer(serializers.ModelSerializer):
    class Meta:
        model = PhotoData
        fields = ("id", "thisdata")


class PhotoSerializer(serializers.ModelSerializer):
    image_data = PhotoDataSerializer()  # or serializers.RelatedField

    class Meta:
        model = Photo
        fields = ("id", "user", "image_data", "test")

and now define a specific method in your api's view that you will be able to use to set the data for any photo

from rest_framework import viewsets, routers, generics
from rest_framework.decorators import action
from rest_framework.response import Response
from rest_framework import status

# ViewSets define the view behavior.


class PhotoViewSet(viewsets.ModelViewSet):
    model = Photo
    serializer_class = PhotoSerializer

    @action(methods=['PATCH'])
    def set_photodata(self, request, pk=None):
        photo = self.get_object()
        serializer = PhotoDataSerializer(data=request.DATA)
        if serializer.is_valid():
            if not photo.image_data:
                photo.image_data = PhotoData.objects.create()
                photo.save()
            photo.image_data.thisdata = serializer.data
            photo.image_data.save()
            return Response({'status': 'ok'})
        else:
            return Response(serializer.errors,
                            status=status.HTTP_400_BAD_REQUEST)

Now you can do almost the same request as you doing now, but you have much more extensibility and division of responsibilities in code. See the URL, it's appended when you have @action's wrapped method.

payload = {"thisdata": '{"test": "ok"}'}
resp = requests.patch(request.build_absolute_uri("/api/photo/1/set_photodata/"), payload)

Hope this helps.