Django template rendering speed

Question

i'm having a performance issue while rendering a django template with a prefetch queryset witch is not really large (~1000 objects in production environment / ~200 objects in developement environment) but has several level of nesting.

Here is the prefetch :

stories = Story.objects.select_related(
            'commande',
            'commande__societe',
            'commande__plateforme',
            'client',).prefetch_related(
            Prefetch('crm_message_related',
                     queryset=Message.objects.select_related(
                         'societe',
                         'plateforme',
                         'type_message').order_by('-date_received')),
            Prefetch('commande__crm_feedback_related'),
            Prefetch('commande__crm_guarantee_related'),
            Prefetch('commande__soyouz_item_related',
                     queryset=Item.objects.select_related(
                         'etat', 'produit', 'produit__produit_enhanced', )),
            Prefetch('commande__tagged_items__tag'),
            Prefetch('commande__soyouz_item_related__tagged_items__tag'),
            Prefetch('commande__soyouz_item_related__soyouz_annulationclient_related'),
            Prefetch('commande__soyouz_item_related__soyouz_achat_related'),
            Prefetch('commande__soyouz_item_related__soyouz_achat_related__soyouz_receptionachat_related'),
            Prefetch('commande__soyouz_item_related__soyouz_achat_related__soyouz_annulationachat_related'),
            Prefetch('soyouz_action_related'),
            Prefetch('soyouz_action_related__receptionmessage__message',
                     to_attr='receptionmessages',
                     queryset=Message.objects.order_by(
                         '-date_received').select_related(
                         'societe', 'type_message', 'plateforme'))
        ).order_by(
            '-commande__date_commande',
            '-date_created'
        ).all().distinct()

The SQL run smoothly, thats not the issue.

The issue is when i try to render my template witch is basicaly

{% for story in stories %}
    {% with items=story.commande.soyouz_item_related.all %}
    {% for item in items %}
        {% for tag in item.tagged_items.all %}
            <button>{{ tag.tag }}</button>
        {% endfor %}
        {{ item.titre|safe|truncatechars:50 }}
    {% endfor %}
{% endfor %}

It takes about 10 sec to display the page in production environment.

I profiled it in my development environment with django-template-timing (https://github.com/orf/django-debug-toolbar-template-timings) and indeed:

name            Times  Tot Time    Queries   Queries Time
stories.html    1      2814,1 ms   0         0 ms

3 seconds for 3 loops with 200 data (in development environment) ? It seems a lot.

Any ideas to improve the speed of template rendering ?

Thanks a lot!

Answer 1

Although django-template-timing reports the rendering time to be 2.8s, I suspect most of the time is consumed in executing the complex SQL query.

In Django, QuerySets are evaluated lazily. This means, the statement you posted Story.objects.select_related(...).distinct() does NOT execute the query in database. The SQL gets executed later. Since you didn't post all the code before rendering the template, I am not sure if the QuerySet gets evalueted before rendering. If not, it may be executed when iterating stories in the template:

{% for story in stories %}

If this is the case, then maybe improving your SQL can reduce the time.

You can check if SQL execution is the culprit by inserting this before rendering the template:

stories = [story for story in stories]

This iteration gets the SQL executed before rendering. After this, if django-template-timing reports a much shorter rendering time, you know SQL is the problem.

If it is indeed a template-rendering performance issue, there are some alternatives:

1. Use a more performant template engine, like Jinja2.
2. Chances are there is room for improving rendering performance with Django template: https://stackoverflow.com/a/26592207/954376.