Why do django templates render dictionaries so slowly?

Question

When I render a moderately complex dictionary (4 levels deep, ~2K data points) using django 1.4's default template system the template rendering step takes over 2800ms. When I do html-gen directly from python instead it takes ~80ms. Even using another template library (jinja2) renders the same data (in fact, almost exactly the same template syntax - as jinja2 is nearly a drop-in replacement) in under 300ms.

Interestingly, you don't even have to actually render the dictionary in the template to cause this performance issue in django's template system... all you have to do is pass it as an available variable to a template. A friend of mine suggested this may mean the system is, "...doing a defensive copy or (more stupidly) a comprehension [which] will take time due to running constructors"

Anyone know why django's default template system takes so long to render dictionaries?

* I'll work on adding requested details below *

I am running in debug mode and have the DebugToolbarMiddleware set as one of my middleware classes. My settings.py file includes:

TEMPLATE_CONTEXT_PROCESSORS = global_settings.TEMPLATE_CONTEXT_PROCESSORS + (
    'django.core.context_processors.request',
)

and....

# rendering like this
return render(
    request,
    template_name='ltm/search_results.html',
    context_instance=RequestContext(request, {
        'menus': menus,
        'results': result_dict
    })
)

Answer 1

It would be great if you could provide us with your template code to better understand what kind of template processing we're dealing with exactly.

Firstly, there are potential differences between how your traversing the contents of the dictionary. dict.items() returns a list of tuples which consumes additional memory and takes time to initially construct, but it's faster to access keys and values than through a generator if you used dict.iteritems().

There's also some overhead involved when you pass in variable names preceded by a dot ., e.g. foo.bar.baz, where Django's template performs what's called a variable lookup, trying to determine whether your accessing a dictionary item by key, an object's attribute or a list item by index. I haven't used Jinja2, so the issue with variable lookups might be completely unrelated, but it is something worth considering if there's a difference between the two.

In either case, since your dealing with a fairly large dictionary, it might help if you could reorganize it's structure in the view to simplify how the data is accessed later in the template.