How to implement a paginator that doesn't call count(*)

Question

I am working on a django website that has a MySQL innodb backend. We have hundreds of thousands of records in several of our tables and this is causing some site stability/performance issues in the admin. Specifically, django likes to make count(*) queries when creating the paginators, and this is causing lots of problems.

With Django 1.3.x, they started to allow for custom pagination classes to be provided. So, I'm interested in finding a way to appropriately speed up or eliminate these queries. So far, I've been looking at these two pages: http://code.google.com/p/django-pagination/source/browse/trunk/pagination/paginator.py https://gist.github.com/1094682 and have not really found them to be what I'm looking for. Any suggestions, help, ect. would be much appreciated.

Answer 1

You can define _count variable in your paginator

  paginator = Paginator(QuerySet, 300)
  paginator._count = 9000 # or use some query here

And here is the part of django paginator code to help you understand what this variable do and how page count works

def _get_count(self):
    "Returns the total number of objects, across all pages."
    if self._count is None:
        try:
            self._count = self.object_list.count()
        except (AttributeError, TypeError):
            # AttributeError if object_list has no count() method.
            # TypeError if object_list.count() requires arguments
            # (i.e. is of type list).
            self._count = len(self.object_list)
    return self._count
count = property(_get_count)

Answer 2

You can pass the count by yourself

paginator = Paginator(models ,25 )
paginator.count=100000

or if you want the exact count you can use

count=int(model.objects.latest('id').id)
paginator = Paginator(models ,25 )
paginator.count=count