Do Django fetches ALL database objects for the paginator function and make it inefficient?

Question

from django.core.paginator import Paginator
from .model                import blog

b = blog.objects.all()
p = Paginator(b, 2)

The above code is from the official Django Pagination Documentation. Can somebody please explain me how this is not inefficient If I have a table with many records? Doesn't the above code fetch everything from the db and then just chops it down? Destroying the purpose of Pagination...

Answer 1

Can somebody please explain me how this is not inefficient If I have a table with many records?

QuerySets are lazy. The blog.objects.all() will not retrieve the records from the database, unless you "consume" the QuerySet. But you did not do that, and so will the paginator not do that either. You "consume" a queryset by iterating over it, calling len(..) over it, converting it to a str(..)ing, and some other functions. But as long as you do not do that. The QuerySet is just an object that represents a "query" that Django will perform if necessary.

The Paginator will construct a QuerySet that is a sliced variant of the queryset you pass. It will first call queryset.count() to retrieve the number of objects, and thus make a SELECT COUNT(*) FROM … query (where we thus do not retrieve the elements itself), and then use a sliced variant that looks like queryset[:2] that will thus look like SELECT … FROM … LIMIT 2. The count is performed to determine the number of pages that exist.

In short, it will thus construct new queries, and perform these on the database. Normally these limit the amount of data returned.