Django - source in serializers leading to too many database calls

Question

I am trying to create a song-artist-album relationship in Django. I have the following models:

class Artist(models.Model):
    gid = models.CharField(max_length=63, blank=True)
    name = models.CharField(max_length=255, blank=True)
    begin_life = models.CharField(max_length=31, blank=True)
    end_life = models.CharField(max_length=31, blank=True)
    type = models.CharField(max_length=1, blank=True)
    gender = models.CharField(max_length=1, blank=True)

class Song(models.Model):
    gid = models.CharField(max_length=63, blank=True)
    title = models.CharField(max_length=255, blank=True)
    artist = models.ForeignKey('Artist', related_name='songs_artist')
    album = models.ForeignKey('Album', related_name='songs_album')
    length = models.IntegerField(default=0)

I have created my ArtistSerializer so that I can retrieve all the songs of the artist when I get the info of any particular artist. This is the serializer I have created:

class ArtistSerializer(serializers.ModelSerializer):
    songs_artist = SongSerializer(source='songs_artist')
    class Meta:
        model = Artist
        fields = ('name', 'type', 'gender', 'begin_life', 'end_life', 'songs_artist')

class SongSerializer(serializers.ModelSerializer):
    artist = SongArtistSerializer()
    album = SongAlbumSerializer()
    class Meta:
        model = Song
        fields = ('id', 'title', 'artist', 'album', 'length')

class SongArtistSerializer(serializers.ModelSerializer):
    class Meta:
        model = Artist
        fields = ('id', 'name')

A quick profiling on the GET method of my artist revealed some troubling facts. Following are the results of the profiling ordered by time and number of calls: http://pastebin.com/bwcKsn2i.

But, when I removed the songs_artist field from my serializer, following was the output of the profiler: http://pastebin.com/0s5k4w7i.

If I read right, the database is being hit 1240 times when I use source!

Is there any other alternative to doing this?

Thanks in advance.

Answer 1

Django REST Framework will not optimize your queries for you, it's up to you to decide how best to remove any N+1 queries. You should follow the guidelines covered in the Django documentation to handle performance issues.

In the case of ForeignKey relationships, you should use select_related in your query, which will pre-fetch the objects in the original query.

In the case of ManyToMany and GenericForeignKey relationships, you should use prefetch_related. I've written quite a bit about this in another Stack Overflow answer, but the gist is that you use it similar to select_related.

You should override the query in get_queryset on the view for best results, as you don't need to worry about Django REST Framework incorrectly cloning the queryset when used as an attribute on the class.