How to get ALL undefined variables from a Jinja2 template?

Question

I am trying to get all undefined variables from a Jinja2 template. Assume that I have a template like below.

tmpstr = """
{% for row in csv %}
sample {{row.field1}} stuff {{row.field2}} morestuff {{row.field3}}
{% endfor %}
"""

and input dictionary as below

cxt = {'csv': [
    {'field3': 1234, 'field4': 12314},
    {'field3': 2222, 'field4': 1213}
]}

Here is how I try to render it.

env = Environment(undefined=Undefined)
tmp = env.from_string(tmpstr)
tmpsrc = tmp.render(cxt)
print(tmpsrc)

Template expect variables field1, field2 and field3 to be present. However field1 and field2 are not present. My aim is to find all missing variables.

Jinja2 silently ignores missing variables. Therefore I tried to add StrictUndefined option:

errs = []
try:
    env = Environment(undefined=StrictUndefined)
    tmp = env.from_string(tmpstr)
    tmpsrc = tmp.render(cxt)
except Exception as e:
    errs.append(str(e))
print(errs)

However this time jinja2 complains about only the first missing variable which is field1.

Therefore I tried another option which is DebugUndefined. This option does not raise an exception and leaves missing variables placeholder in the template output untouched. Thus I can not collect missing variables.

Can you please suggest how can I get missing variables in a jinja2 template?

Here is runnable code if anyone wants to try it out:

from jinja2 import BaseLoader,Environment,StrictUndefined,DebugUndefined,Undefined
tmpstr = """
{% for row in csv %}
sample {{row.field1}} stuff {{row.field2}} morestuff {{row.field3}}
{% endfor %}
"""
cxt = {'csv': [
    {'field3': 1234, 'field4': 12314},
    {'field3': 2222, 'field4': 1213}
]}
env = Environment(undefined=Undefined)
tmp = env.from_string(tmpstr)
tmpsrc = tmp.render(cxt)
print('CASE 1: undefined=Undefined')
print(tmpsrc)

errs = []
try:
    env = Environment(undefined=StrictUndefined)
    tmp = env.from_string(tmpstr)
    tmpsrc = tmp.render(cxt)
except Exception as e:
    errs.append(str(e))
print('CASE 2: undefined=StrictUndefined')
print(errs)

errs = []
try:
    env = Environment(undefined=DebugUndefined)
    tmp = env.from_string(tmpstr)
    tmpsrc = tmp.render(cxt)
except Exception as e:
    errs.append(str(e))

print('CASE 3: undefined=DebugUndefined')
print(errs)
print(tmpsrc)

Answer 1

Using find_undeclared_variables with DebugUndefined you can properly raise an exception mentioning all variables that are missing:

import jinja2
from jinja2.meta import find_undeclared_variables

env = jinja2.Environment(undefined=jinja2.DebugUndefined)
template = env.from_string('foo={{ foo }}, bar={{ bar}}, baz={{ baz }}')

# Render template without passing all variables
rendered = template.render(foo=1)

# Check if rendering was done correctly
ast = env.parse(rendered)
undefined = find_undeclared_variables(ast)  # {'bar', 'baz'}
if undefined:
    raise jinja2.UndefinedError(f'The following variables are undefined: {undefined!r}')

If you prefer logging, you can replace the exception raising with your own logging calls using the contents of undefined.

PS: I am relatively new to Jinja, but I am quite surprised this is not the default behavior of env.render. I wonder why authors/maintainers think having missing variables silently ignored by default being a good thing...

Answer 2

I found the solution for your question, using jinja2.make_logging_undefined. I was in the same boat as you are and have been search high and low for an answer. Most of the answer pointing me to use parsed templates, however I couldn't figure out how to get the context into the parsed templates.

I finally able to make this work using the make_logging_undefined. If you want to find all the undefined variables, make sure to use just the Undefined base class rather than StrictUndefined. Using StrictUndefined will cause jinja to throw exception at the first encounter of the undefine.

Just a disclaimer: I'm not an python nor jinja expert, so the code is not the most efficient nor structured. But it serves my purpose. This is just POC code.

Here's the code:

import jinja2
import logging
from jinja2 import Environment, Undefined
from jinja2.exceptions import UndefinedError

def main():
    templateLoader = jinja2.FileSystemLoader( searchpath="D:\\somelocation\\" )

    logging.basicConfig()
    logger = logging.getLogger('logger')
    LoggingUndefined = jinja2.make_logging_undefined(logger=logger,base=jinja2.Undefined)

    templateEnv = jinja2.Environment( loader=templateLoader, undefined=LoggingUndefined)

    TEMPLATE_FILE = "./example1.jinja"

    template = templateEnv.get_template( TEMPLATE_FILE )

    FAVORITES = [ "chocolates", "lunar eclipses", "rabbits" ]
    # Specify any input variables to the template as a dictionary.
    templateVars = { "title" : "Test Example",
                     "description" : "A simple inquiry of function.",
                     "favorites" : FAVORITES,
                     "whatever" : "1"
                   }    
    # Finally, process the template to produce our final text.
    try:
        outputText = template.render( templateVars )
    except ( UndefinedError) as err:
        print err

if __name__ == '__main__':
    main()

example1.jinja:

<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8" />

  <title>{{ title }}</title>
  <meta name="description" content="{{ description }}" />
</head>

<body>

<div id="content">
  <p>Greetings visitor!  These are a list of my favorite things:</p>

  <ul>
  {% for item in favorites %}
    <li>{{ item }}</li>

  <li>My favorites: {{ favorites[1] }} </li>
  {% endfor %}
  {{ undefined_var1 }}
  {{ underfined_var2 }}
  </ul>
</div>

</body>
</html>

Here's the sample output:

WARNING:logger:Template variable warning: undefined_var1 is undefined
WARNING:logger:Template variable warning: underfined_var2 is undefined