Django Rest Framework "This field is required" only when POSTing JSON, not when POSTing form content

Question

I'm getting a strange result whereby POSTing JSON to a DRF endpoint returns:

{"photos":["This field is required."],"tags":["This field is required."]}'

Whereas when POSTing form data DRF doesn't mind that the fields are empty.

My model is:

class Story(CommonInfo):
    user = models.ForeignKey(User)
    text = models.TextField(max_length=5000,blank=True)
    feature = models.ForeignKey("Feature", blank=True, null=True)
    tags = models.ManyToManyField("Tag")

My serializer is:

class StorySerializer(serializers.HyperlinkedModelSerializer):
    user = serializers.CharField(read_only=True) 

    def get_fields(self, *args, **kwargs):
        user = self.context['request'].user
        fields = super(StorySerializer, self).get_fields(*args, **kwargs)
        fields['feature'].queryset = fields['feature'].queryset.filter(user=user)
        fields['photos'].child_relation.queryset = fields['photos'].child_relation.queryset.filter(user=user)
        return fields

    class Meta:
        model = Story
        fields = ('url', 'user', 'text', 'photos', 'feature', 'tags')

And my api.py is:

class StoryViewSet(viewsets.ModelViewSet):
    serializer_class = StorySerializer

    def get_queryset(self):
        return self.request.user.story_set.all()

    def perform_create(self, serializer):
        serializer.save(user=self.request.user)

The results:

# JSON request doesn't work
IN: requests.post("http://localhost:8001/api/stories/",
               auth=("user", "password",),
               data=json.dumps({'text': 'NEW ONE!'}),
               headers={'Content-type': 'application/json'}
              ).content
OUT: '{"photos":["This field is required."],"tags":["This field is required."]}'

# Form data request does work
IN: requests.post("http://localhost:8001/api/stories/",
               auth=("user", "password",),
               data={'text': 'NEW ONE!'},
              ).content
OUT: '{"url":"http://localhost:8001/api/stories/277/","user":"user","text":"NEW ONE!","photos":[],"feature":null,"tags":[]}'

Answer 1

The issue here isn't obvious at first, but it has to do with a shortcoming in form-data and how partial data is handled.

Form data has two special cases that Django REST framework has to handle

1. There is no concept of "null" or "empty" data for some inputs, including checkboxes and other inputs that allow for multiple selections.
2. There is no input type that supports multiple values for a single field, checkboxes being the one exception.

Both of these combine together to make it difficult to handle accepting form data within Django REST framework, so it has to handle a few things differently from most parsers.

1. If a field is not passed in, it is assumed to be None or the default value for the field. This is because inputs with no values are not passed along in the form data, so their key is missing.
2. If a single value is passed in for a multiple-value field, it will be treated like the one selected value. This is because there is no difference between a single checkbox selected out of many and a single checkbox at all in form data. Both of them are passed in as a single key.

But the same doesn't apply to JSON. Because you are not passing an empty list in for the photos and tags keys, DRF does not know what to give it for a default value and does not pass it along to the serializer. Because of this, the serializer sees that there is nothing passed in and triggers the validation error because the required field was not provided.

So the solution is to always provide all keys when using JSON (not including PATCH requests, which can be partial), even if they contain no data.