Ansible loop over glob

Question

I am trying to create a super complicated, but cool data structure for variables.

defaults/main.yml:

docker_compose_keycloak_config_cli_keycloak_url: http://keycloak:8080
docker_compose_keycloak_config_cli_user: admin
docker_compose_keycloak_config_cli_user_password: ~
docker_compose_keycloak_config_cli_client_id: admin-cli                                                                                                                                                                       
docker_compose_keycloak_config_cli_clientsecret: ~                        

Template using those variables

#templates/env-vars-keycloak-config-cli
#jinja2: lstrip_blocks: "True"
{% if docker_compose_keycloak_config_cli_keycloak_url %}
KEYCLOAK_URL={{ docker_compose_keycloak_config_cli_keycloak_url }}
{% endif %}
{% if docker_compose_keycloak_config_cli_user %}                                                         
KEYCLOAK_USER={{ docker_compose_keycloak_config_cli_user }}
{% endif %}
{% if docker_compose_keycloak_config_cli_user_password %}
KEYCLOAK_PASSWORD={{ docker_compose_keycloak_config_cli_user_password }}
{% endif %}
{% if docker_compose_keycloak_config_cli_client_id %}
KEYCLOAK_CLIENTID={{ docker_compose_keycloak_config_cli_client_id }}
{% endif %}
{% if docker_compose_keycloak_config_cli_clientsecret %}
KEYCLOAK_CLIENTSECRET={{ docker_compose_keycloak_config_cli_clientsecret }}
{% endif %}
{% if docker_compose_keycloak_config_cli_granttype %}
KEYCLOAK_GRANTTYPE={{ docker_compose_keycloak_config_cli_granttype}}
{% endif %}
{% if docker_compose_keycloak_config_cli_loginrealm %}
KEYCLOAK_LOGINREALM={{ docker_compose_keycloak_config_cli_loginrealm }}
{% endif %}
{% if docker_compose_keycloak_config_cli_sslverify %}
KEYCLOAK_SSLVERIFY={{ docker_compose_keycloak_config_cli_sslverify }}
{% endif %}
{% if docker_compose_keycloak_config_cli_httpproxy %}
KEYCLOAK_HTTPPROXY={{ docker_compose_keycloak_config_cli_httpproxy }}
{% endif %}
{% if docker_compose_keycloak_config_cli_connecttimeout %}
KEYCLOAK_CONNECTTIMEOUT={{ docker_compose_keycloak_config_cli_connecttimeout }}
{% endif %}
{% if docker_compose_keycloak_config_cli_readtimeout %}
KEYCLOAK_READTIMEOUT={{ docker_compose_keycloak_config_cli_readtimeout }}
{% endif %}
{% if docker_compose_keycloak_config_cli_availabilitycheck_enabled %}
KEYCLOAK_AVAILABILITYCHECK_ENABLED={{ docker_compose_keycloak_config_cli_availabilitycheck_enabled }}
{% endif %}
{% if docker_compose_keycloak_config_cli_availabilitycheck_timeout %}                                                                                                                                                         
KEYCLOAK_AVAILABILITYCHECK_TIMEOUT={{ docker_compose_keycloak_config_cli_availabilitycheck_timeout }}                                                                                                                         
{% endif %}                                                                                                                                                                                                                                                                                                                                                                                                  

Now, for environment variables you don't really want to add null variables (~).

It would be cool to do something like this:

#templates/env-vars-keycloak-config-cli
#jinja2: lstrip_blocks: "True"
{% for key, value in docker_compose_keycloak_config_cli_envvar_* %}
{% if value -%}
{{key}}={{value}}
{% endif -%}
{% endfor %}

This would make my template super short and I don't have to waste time typing so many variables.

Plus, I don't have to rely on a single variable that holds the environment variables. I could edit host_vars or group_vars with only the one specific variable and not rewrite the big variable.

I.e. make my defaults/main.yml looks like this:

docker_compose_keycloak_config_cli_envvar_keycloak_url: 
  KEYCLOAK_URL: http://keycloak:8080

I am testing out how I could loop these variables.

E.g.

- name: Create list of vars
  debug:
    msg: "{{ item }}"
  loop: "{{ docker_compose_keycloak_config_cli_envvar_* }}"
  tags: test

This gave me the error:

fatal: [login.cyber.ee]: FAILED! => {"msg": "template error while templating string: unexpected 'end of print statement'. String: {{ docker_compose_keycloak_config_envvar_* }}"}

How could I gather these type of variables with the glob?

Answer 1

Ease your own life and simplify / DRY your variables.

Make your defaults/main.yml looks like this, where you don't have to keep on repeating docker_compose_keycloak_config_cli for every single dictionary key:

docker_compose_keycloak_config_cli:
  keycloak_url: http://keycloak:8080
  user: admin
  password: ~
  client_id: admin-cli
  clientsecret: ~
  envvar:
    keycloak_url: http://keycloak:8080

Then your requirement becomes trivial:

{% for key, value in docker_compose_keycloak_config_cli.envvar.items() %}
{{ key | upper }}={{ value }}
{% endfor %}

Which would give you:

KEYCLOAK_URL=http://keycloak:8080

---

And even better to reduce duplication, use YAML anchors:

docker_compose_keycloak_config_cli:
  keycloak_url: &keycloak_url http://keycloak:8080
  user: admin
  password: ~
  client_id: admin-cli
  clientsecret: ~
  envvar:
    keycloak_url: *keycloak_url

---

In order to allow overriding it, recursively combine the overriding dictionary into the existing variable. Then use it as described above.

- set_fact:
    docker_compose_keycloak_config_cli: >-
      {{
        docker_compose_keycloak_config_cli
        | combine(
          docker_compose_keycloak_config_cli_override | default({}),
          recursive=True
        )
      }}

---

The two tasks:

- set_fact:
    docker_compose_keycloak_config_cli: >-
      {{
        docker_compose_keycloak_config_cli
        | combine(
          docker_compose_keycloak_config_cli_override | default({}),
          recursive=True
        )
      }}
  vars:
    docker_compose_keycloak_config_cli:
      keycloak_url: &keycloak_url http://keycloak:8080
      user: admin
      password: ~
      client_id: admin-cli
      clientsecret: ~
      envvar:
        keycloak_url: *keycloak_url
        other_env: foo
    docker_compose_keycloak_config_cli_override:
      envvar:
        keycloak_url: http://keycloak:80

- debug:
    var: docker_compose_keycloak_config_cli

Would yield:

docker_compose_keycloak_config_cli:
  client_id: admin-cli
  clientsecret: null
  envvar:
    keycloak_url: http://keycloak:80
    other_env: foo
  keycloak_url: http://keycloak:8080
  password: null
  user: admin