Substituting variables in Ansible playbook for EC2 security group rules

Question

I am attempting to use Ansible to provision AWS EC2 security groups.

I want some of these groups to have multiple rules. Some rules will apply only to my internal VPC while others are open to the world.

I would like to create the security groups in a loop that reads configuration from a list variable, but I don't want to hard code the CIDR for the internal VPC. I would rather get the CIDR from my VPC facts, but I haven't found a satisfactory way to substitute the CIDR into the rule.

To make this more clear, here is a (contrived) example. My original list of groups had all of the CIDRs hard coded:

aws_security_groups:
  - name: Webservers
    description: Security group for webservers
    region: my_aws_region
    rules: 
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: 0.0.0.0/0
      - proto: tcp
        from_port: 80
        to_port: 80
        cidr_ip: 0.0.0.0/0
  - name: Databases
    description: Security group for internal database access
    region: my_aws_region
    rules: 
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: 0.0.0.0/0
      - proto: tcp
        from_port: 3306
        to_port: 3306
        cidr_ip: <vpc.cidr.hard.coded/16>

The original play works and is very simple:

- name: Provision EC2 security groups
  ec2_group:
    name: "{{ item.name }}"
    description: "{{ item.description }}"
    region: "{{ item.region }}"
    state: present
    rules: "{{ item.rules }}"
  with_items: "{{ aws_security_groups }}"

I can add or subtract rules and know that the groups will be synced. However, I don't want to hard code the CIDRs... I want my list of groups to look like:

aws_security_groups:
  - name: Webservers
    description: Security group for webservers
    region: my_aws_region
    rules: 
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: all
      - proto: tcp
        from_port: 80
        to_port: 80
        cidr_ip: all

  - name: Databases
    description: Security group for internal database access
    region: my_aws_region
    rules: 
      - proto: tcp
        from_port: 22
        to_port: 22
        cidr_ip: all
      - proto: tcp
        from_port: 3306
        to_port: 3306
        cidr_ip: internal

My current strategy is to use with_subelements to run a bunch of ec2_group commands for each rule individually:

- name: Gather EC2 VPC facts
  ec2_vpc_net_facts:
    region: my_aws_region
  register: vpcs

- name: Provision EC2 security groups
  ec2_group:
    name: "{{ item.0.name }}"
    description: "{{ item.0.description }}"
    region: "{{ item.0.region }}"
    state: present
    purge_rules: false
    rules: 
      - from_port: "{{ item.1.from_port }}"
        to_port: "{{ item.1.to_port }}"
        proto: "{{ item.1.proto }}"
        cidr_ip: "{{ (item.1.cidr_ip == 'internal') | ternary(vpcs.vpcs.0.cidr_block, (item.1.cidr_ip == 'all') | ternary('0.0.0.0/0', item.1.cidr_ip)) }}"
  with_subelements: 
    - "{{ aws_security_groups }}"
    - rules

The substitutions work, but I am stuck with some uncomfortable trade offs.

First, I have to turn off purge_rules since every loop will just erase the previous rules if I don't. Because of this, if I make changes to the rules I may be stuck with old rules that I will need to keep track of and clean up.

Second, the nested ternaries are awkward and not easily expandable.

Third, I'm making multiple calls when one would suffice.

I feel like I am missing something obvious or that I am over-complicating this. How can I accomplish the CIDR substitution, or more generally, any substitution of this kind in an Ansible playbook? Is substitution preferable or is there a better method to accomplish the same task?

Any help on this is greatly appreciated.

Answer 1

The following way you avoid purge_rules problem and it's cleaner, imho.

Tasks:

- set_fact:
    from_template: "{{ lookup('template', './template.j2') }}"
  vars:
    to_template_aws_security_groups: "{{ aws_security_groups }}"
    to_template_vpcs: "{{ vpcs }}"

- name: Provision EC2 security groups
  ec2_group:
    name: "{{ item.name }}"
    description: "{{ item.description }}"
    region: "{{ item.region }}"
    state: present
    rules: "{{ item.rules }}"
  with_items: "{{ from_template.aws_security_groups }}"

template.j2:

{
  "aws_security_groups": [
    {% for aws_security_group in to_template_aws_security_groups %}
    {
      "description": "{{ aws_security_group.description }}",
      "name": "{{ aws_security_group.name }}",
      "region": "{{ aws_security_group.region }}",
      "rules": [
        {% for rule in aws_security_group.rules %}
        {
          "cidr_ip": "{{ (rule.cidr_ip == 'internal') | ternary(to_template_vpcs.vpcs.0.cidr_block, (rule.cidr_ip == 'all') | ternary('0.0.0.0/0', rule.cidr_ip)) }}",
          "from_port": "{{ rule.from_port }}",
          "proto": "{{ rule.proto }}",
          "to_port": {{ rule.to_port }}
        }{% if not loop.last %},{% endif %}
        {% endfor %}
      ]
    }{% if not loop.last %},{% endif %}
    {% endfor %}
  ]
}