Provisioning multiple spot instances with instance_interruption_behavior using Ansible

Question

I know it seems to relate to other questions asked in the past but I feel it's not. You will be the judge of that.

I've been using Ansible for 1.5 years now and I'm aware it's not the best tool to provision infrastructure , Yet for my needs it suffice.

I'm using AWS as my cloud provider

Is there any way to cause Ansible to provision multiple spot instances of the same type( similar to the count attribute in the ec2 ansible module) with the instance_interruption_behavior set to stop(behavior) instead of the default terminate?

My Goal:

Set up multiple EC2 instances with different ec2_spot_requests to Amazon. ( spot request per instance) But instead of using the default instance_interruption_behavior which is terminate, I wish to set it to stop(behavior )

What I've done so far:

I'm aware that Ansible has the following module to handle ec2 infrastructure . Mainly the

ec2. https://docs.ansible.com/ansible/latest/modules/ec2_module.html

Not to be confused with the ec2_instance module.

https://docs.ansible.com/ansible/latest/modules/ec2_instance_module.html

I could use the ec2 module to set up spot instances , As I've done so far, as follows:

ec2:
  key_name: mykey
  region: us-west-1
  instance_type: "{{ instance_type }}"
  image: "{{ instance_ami }}"
  wait: yes
  wait_timeout: 100
  spot_price: "{{ spot_bid }}"
  spot_wait_timeout: 600
  spot_type: persistent
  group_id: "{{ created_sg.group_id }}"
  instance_tags:
    Name: "Name"
  count: "{{ my_count }}"
  vpc_subnet_id: " {{ subnet }}"
  assign_public_ip: yes
  monitoring: no
  volumes:
    - device_name: /dev/sda1
      volume_type: gp2
      volume_size: "{{ volume_size }}"

Pre Problem:

The above sample module is nice and clean and allow me to setup spot instance according to my requirements , by using the ec2 module and the count attribute.

Yet, It doesn't allow me to set the instance_interruption_behavior to stop( unless I'm worng), It just defaults to terminate.

Other modules:

To be released on Ansible version 2.8.0 ( Currently only available in develop) , There is a new module called: ec2_launch_template

https://docs.ansible.com/ansible/devel/modules/ec2_launch_template_module.html

This module gives you more attributes , which allows you ,with the combination of ec2_instance module to set the desired instance_interruption_behavior to stop(behavior). This is what I did:

- hosts: 127.0.0.1
  connection: local
  gather_facts: True
  vars_files:
    - vars/main.yml
  tasks:
    - name: "create security group"
      include_tasks: tasks/create_sg.yml

    - name: "Create launch template to support interruption behavior 'STOP' for spot instances"
      ec2_launch_template:
        name: spot_launch_template
        region: us-west-1
        key_name: mykey
        instance_type: "{{ instance_type }}"
        image_id: "{{ instance_ami }}"
        instance_market_options:
          market_type: spot
          spot_options:
            instance_interruption_behavior: stop
            max_price: "{{ spot_bid }}"
            spot_instance_type: persistent
        monitoring:
          enabled: no
        block_device_mappings:
          - device_name: /dev/sda1
            ebs:
              volume_type: gp2
              volume_size: "{{ manager_instance_volume_size }}"
      register: my_template

- name: "setup up a single spot instance"
  ec2_instance:
    instance_type: "{{ instance_type }}"
    tags:
      Name: "My_Spot"
    launch_template:
      id: "{{ my_template.latest_template.launch_template_id }}"
    security_groups:
      - "{{ created_sg.group_id }}"

The main Problem:

As it shown from the above snippet code, the ec2_launch_template Ansible module does not support the count attribute and the ec2_instance module doesn't allow it either.

Is there any way cause ansible to provision multiple instances of the same type( similar to the count attribute in the ec2 ansible module)?

Answer 1

In general if you are managing lots of copies of EC2 instances at once you probably want to either use Auto-scaling groups or EC2 Fleets.

There is an ec2_asg module that allows you to manage auto-scaling groups via ansible. This would allow you to deploy multiple EC2 instances based on that launch template.

Another technique to solve this issue would be to use the cloudformation or terraform modules if AWS cloudformation templates or terraform plans support what you want to do. This would also let you use Jinja2 templating syntax to do some clever template generation before supplying the infrastructure templates if necessary.