Google Deployment manager: MANIFEST_EXPANSION_USER_ERROR multiline variables

Question

Trying to use Google Deployment Manager with YAML and Jinja with a multi-line variables, such as:

startup_script_passed_as_variable: |
  line 1
  line 2
  line 3

And later:

{% if 'startup_script_passed_as_variable' in properties %}
    - key: startup-script
      value: {{properties['startup_script_passed_as_variable'] }}
{% endif %}

Gives MANIFEST_EXPANSION_USER_ERROR:

ERROR: (gcloud.deployment-manager.deployments.create) Error in Operation operation-1432566282260-52e8eed22aa20-e6892512-baf7134:

MANIFEST_EXPANSION_USER_ERROR
Manifest expansion encountered the following errors: while scanning a simple key in "" could not found expected ':' in ""

Tried (and failed):

{% if 'startup_script' in properties %}
        - key: startup-script
          value: {{ startup_script_passed_as_variable }}
{% endif %}

also

{% if 'startup_script' in properties %}
        - key: startup-script
          value: | 
            {{ startup_script_passed_as_variable }}
{% endif %}

and

{% if 'startup_script' in properties %}
        - key: startup-script
          value: | 
            {{ startup_script_passed_as_variable|indent(12) }}
{% endif %}

Answer 1

The problem is the combination of YAML and Jinja. Jinja escapes the variable but fails to indent it as YAML would require when being passed as a variable.

Related: https://github.com/saltstack/salt/issues/5480

Solution: Pass the multi-line variable as an array

startup_script_passed_as_variable: 
    - "line 1"
    - "line 2"
    - "line 3"

The quoting is important if your value starts with # (which startup script on GCE does, ie #!/bin/bash) since it will be treated as a comment otherwise.

{% if 'startup_script' in properties %}
        - key: startup-script
          value: 
{% for line in properties['startup_script'] %}
            {{line}}
{% endfor %}
{% endif %}

Putting it here since there aren't much Q&A material for Google Deployment manager.

Answer 2

There is no clean way to do this in Jinja. As you yourself have pointed out, because YAML is a whitespace-sensitive language, it is difficult to effectively template around.

One possible hack is to split the string property into a list and then iterate over the list.

For example, providing the property:

startup-script: |
  #!/bin/bash
  python -m SimpleHTTPServer 8080

you can use it in your Jinja template:

{% if 'startup_script' in properties %}
      - key: startup-script
      value: |
{% for line in properties['startup-script'].split('\n') %}
        {{ line }}
{% endfor %}

Here is also a full working example of this.

This method will work, but generally cases like this are when people start considering using a python template. Because you are working with an object model in python, you do not have to deal with indentation problems. For example:

'metadata': {
    'items': [{
        'key': 'startup-script',
        'value': context.properties['startup_script']
    }]
}

An example of the python template can be found in the Metadata From File example.