Django Rest Framework pagination extremely slow count

Question

I turned pagination on in Django Rest framework and it appears to be incredibly slow. Count looks like the culprit, and is taking hundreds of milliseconds to return each time due to the millions of rows in the tables.

I am using postgresql as the database. Is there any way to not count the rows and still use pagination? The performance was fine before this was enabled if I manually filtered the queryset.

Answer 1

The issue is, that the query used to count is the same potentially complex one as used to fetch the data. That's rather wasteful. PageNumberPagination uses Django's own Paginator internally.

To make the query for the count simpler override the paginator class DRF uses:

from django.core.paginator import Paginator
from django.utils.functional import cached_property
from rest_framework.pagination import PageNumberPagination

class FasterDjangoPaginator(Paginator):
    @cached_property
    def count(self):
        # only select 'id' for counting, much cheaper
        return self.object_list.values('id').count()


class FasterPageNumberPagination(PageNumberPagination):
    django_paginator_class = FasterDjangoPaginator