Load YAML nested with Jinja2 in Python

Question

I have a YAML file (all.yaml) that looks like:

...
var1: val1
var2: val2
var3: {{var1}}-{{var2}}.txt
...

If I load it in Python like this:

import yaml

f = open('all.yaml')
dataMap = yaml.safe_load(f)
f.close()
print(dataMap["var3"])

the output is {{var1}}-{{var2}}.txt and not val1-val2.txt.

Is it possible to replace the nested vars with the value?

I tried to load it with:

import jinja2
templateLoader = jinja2.FileSystemLoader( searchpath="/path/to/dir" )
templateEnv = jinja2.Environment( loader=templateLoader )
TEMPLATE_FILE = "all.yaml"
template = templateEnv.get_template( TEMPLATE_FILE )

The exception is no longer thrown, now I am stuck and have to research how to proceed.

Answer 1

First define an Undefined class and load yaml to get known values. Then load it again and render with known values.

#!/usr/bin/env python

import yaml
from jinja2 import Template, Undefined

str1 = '''var1: val1
var2: val2
var3: {{var1}}-{{var2}}.txt
'''

class NullUndefined(Undefined):
  def __getattr__(self, key):
    return ''

t = Template(str1, undefined=NullUndefined)
c = yaml.safe_load(t.render())

print t.render(c)

Run it:

$ ./test.py
var1: val1
var2: val2
var3: val1-val2.txt

Answer 2

Here is one possible solution:

1. Parse your YAML document with the yaml module
2. Iterate over the keys in your YAML document, treating each value as a Jinja2 template to which you pass in the keys of the YAML document as parameters.

For example:

import yaml
from jinja2 import Template

with open('sample.yml') as fd:
    data = yaml.load(fd)

for k, v in data.items():
    t = Template(v)
    data[k] = t.render(**data)

print yaml.safe_dump(data, default_flow_style=False)

This will work fine with your particular example, but wouldn't do anything useful for, say, nested data structures (in fact, it would probably just blow up).