Django - queries made repeat/inefficient

Question

Alright, I have a Django view, like this:

@render_to('home/main.html')
def login(request):
    # also tried Client.objects.select_related().all()
    clients = Client.objects.all()
    return {'clients':clients}

And I have a template, main.html, like this:

<ul>
{% for client in clients %}
<li>{{ client.full_name }}</li>
    <ul>
    {% for pet in client.pets.all %}
        <li>{{ pet.full_name }}</li>
    {% endfor %}
    </ul>
{% endfor %}
</ul>

I also print out all the queries in sql_queries at the bottom of my base template. When I run this view, the following queries are made:

SELECT `home_client`.`id`, ... FROM `home_client`;
SELECT `home_pet`.`id`, ... FROM `home_pet` WHERE `home_pet`.`client_id` = 1;
SELECT `home_client`.`id`, ... FROM `home_client` WHERE `home_client`.`id` = 1;
SELECT `home_client`.`id`, ... FROM `home_client` WHERE `home_client`.`id` = 1;
SELECT `home_pet`.`id`, ... FROM `home_pet` WHERE `home_pet`.`client_id` = 2;
SELECT `home_client`.`id`, ... FROM `home_client` WHERE `home_client`.`id` = 2; 

My question is, why are all these queries being made? Shouldn't it just be 1 query to retrieve all the clients and a query per client to retrieve all the pets from each client? I have 2 clients in the home_client table, so it should be 3 queries total. Most troubling of all is that queries 3 and 4 are 100% identical. I don't want to "prematurely optimize" or anything but I do want to make sure Django isn't being wildly inefficient. Any help on this would be appreciated. Thanks.

Answer 1

Django uses a cache. The RDBMS uses a cache. Don't prematurely optimize the queries.

You can play with bulk queries in your view function instead of one-at-a-time queries in your template.

@render_to('home/main.html')
def login(request):
    # Query all clients 
    clients = Client.objects.all()
    # Assemble an in-memory table of pets
    pets = collections.defaultdict(list)
    for p in Pet.objects.all():
        pets[pet.client].append(p)
    # Create clients and pets tuples
    clientsPetTuples = [ (c,pets[c]) for c in clients ]
    return {'clientPets': clientsPetTuples}

However, you don't seem to have any evidence that your template is the slowest part of your application.

Further, this trades off giant memory use against SQL use. Until you have measurements that prove that your template queries are actually slow, you shouldn't be over thinking the SQL.

Don't worry about the SQL until you have evidence.