How to test ansible roles?

Question

Scenario

I want to develop ansible roles. Those roles should be validated through a CI/CD process with molecule and utilize docker as driver. This validation step should include multiple Linux flavours (e.g. centos/ubuntu/debian) times the supported ansible versions.

The tests should then be executed such that the role is verified with

centos:7 + ansible:2.5
centos:7 + ansible:2.6
centos:7 + ansible:2.7
...
ubuntu:1604 + ansible:2.5
ubuntu:1604 + ansible:2.6
ubuntu:1604 + ansible:2.7
...

---

The issues at hand

1. there is no official ansible image availabe
2. how to best test a role for ansible version compatibility?

---

Issue 1: no official ansible images

The official images by the ansible team have been deprecated for about 3 years now:

• https://hub.docker.com/r/ansible/ubuntu14.04-ansible
• https://hub.docker.com/r/ansible/centos7-ansible

In addition, the link the deprecated images refer to in order to find new images supporting ansible is quite useless due to the sheer amount of results

https://hub.docker.com/search/?q=ansible&page=1&isAutomated=0&isOfficial=0&pullCount=1&starCount=0

Is there a well-maintained ansible docker image by the community (or ansible) that fills the void?

Preferable with multiple versions that can be pulled and a CI process that builds and validates the created image regularly.

Why am I looking for ansible images? I do not want to reinvent the wheel (if possible). I would like to use the images to test ansible roles via molecule for version incompatibility.

I searched but could not find anything truly useful. What images are you using to run ansible in a container/orchestrator? Do you build and maintain the images yourself?

e.g. https://hub.docker.com/r/ansibleplaybookbundle/apb-base/tags

looked promising, however, the images in there are also over 7 months old (at least).

---

Issue 2: how to best test a role for ansible version compatibility?

Is creating docker images for each combination of OS and ansible version the best way to test via molecule and docker as driver? Or is there a smarter way to test backward compatibility of ansible roles with multiple OS times different ansible versions?

I already test my ansible roles with molecule and docker as driver. Those tests currently only testing the functionality of the role on various Linux distros, but not the ansible backward compatibility by running the role with older ansible versions.

Here an example role with travis tests for centos7/ubuntu1604/ubuntu1804 based on geerlingguy's ntp role: https://github.com/Gepardec/ansible-role-ntp

Answer 1

Solution

In order to test ansible roles with multiple versions of ansible, python and various Linux flavors we can use

• molecule for our ansible role functionality
• docker as our abstraction layer on which we run the target system for our ansible role
• tox to setup generic virtualenvs and test our various combinations without side effects
• travis to automate it all

This will be quite a long/detailed answer. You can check out an example ansible role with the whole setup here

• https://github.com/ckaserer/ansible-role-example
• https://travis-ci.com/ckaserer/ansible-role-example

---

Step 1: test ansible role with molecule

Molecule docs: https://molecule.readthedocs.io/en/stable/

Fixes Issue: 1) no official ansible images

I could create ansible images for every distro I would like to test as jeff geerling describes in his blog posts.

The clear downside of this approach: The images will need maintenance (eventually)

However, with molecule, we can combine base images with a Dockerfile.j2 (Jinja2 Template) to create the images with minimal requirements to run ansible. Through this approach, we are now able to use the official linux distro images from docker hub and do not need to maintain a docker repo for each linux distro and the various versions.

Here the important bits in molecule.yml

platforms:
  - name: instance-${TOX_ENVNAME}
    image: ${MOLECULE_DISTRO:-'centos:7'}
    command: /sbin/init
    volumes:
      - /sys/fs/cgroup:/sys/fs/cgroup:ro
    privileged: true

The default dockerfile.j2 from molecule is already good, but I have a few additions

# Molecule managed

{% if item.registry is defined %}
FROM {{ item.registry.url }}/{{ item.image }}
{% else %}
FROM {{ item.image }}
{% endif %}

{% if item.env is defined %}
{% for var, value in item.env.items() %}
{% if value %}
ENV {{ var }} {{ value }}
{% endif %}
{% endfor %}
{% endif %}

RUN if [ $(command -v apt-get) ]; then                                                        apt-get update && apt-get install -y python sudo bash ca-certificates iproute2 && apt-get clean; \
    elif [ $(command -v zypper) ]; then                                                       zypper refresh && zypper install -y python sudo bash python-xml iproute2 && zypper clean -a; \
    elif [ $(command -v apk) ]; then                                                          apk update && apk add --no-cache python sudo bash ca-certificates; \
    elif [ $(command -v xbps-install) ]; then                                                 xbps-install -Syu && xbps-install -y python sudo bash ca-certificates iproute2 && xbps-remove -O; \
    elif [ $(command -v swupd) ]; then                                                        swupd bundle-add python3-basic sudo iproute2; \
    elif [ $(command -v dnf) ] && cat /etc/os-release | grep -q '^NAME=Fedora';          then dnf makecache && dnf --assumeyes install python sudo python-devel python*-dnf bash iproute && dnf clean all; \
    elif [ $(command -v dnf) ] && cat /etc/os-release | grep -q '^NAME="CentOS Linux"' ; then dnf makecache && dnf --assumeyes install python36 sudo platform-python-devel python*-dnf bash iproute && dnf clean all && ln -s /usr/bin/python3 /usr/bin/python; \
    elif [ $(command -v yum) ]; then                                                          yum makecache fast && yum install -y python sudo yum-plugin-ovl bash iproute && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \
    fi

# Centos:8 + ansible 2.7 failed with error: "The module failed to execute correctly, you probably need to set the interpreter"
# Solution: ln -s /usr/bin/python3 /usr/bin/python

By default, this will test the role with centos:7. However, we can set the environment variable MOLECULE_DISTRO to whichever image we would like to test and run it via

MOLECULE_DISTRO=ubuntu:18.04 molecule test

Summary

We use official distro images from docker hub to test our ansible role via molecule.

The files used in this step

• molecule.yml https://github.com/ckaserer/ansible-role-example/blob/master/molecule/default/molecule.yml
• Dockerfile.j2 https://github.com/ckaserer/ansible-role-example/blob/master/molecule/default/Dockerfile.j2

Source

• https://molecule.readthedocs.io/en/stable/examples.html#systemd-container
• https://www.jeffgeerling.com/blog/2019/how-i-test-ansible-configuration-on-7-different-oses-docker
• https://www.jeffgeerling.com/blog/2018/testing-your-ansible-roles-molecule

---

Step 2: Check compatibility for your role ( python version X ansible version X linux distros )

Fixes Issue 2: how to best test a role for ansible version compatibility?

Let's use tox to create virtual environments to avoid side effects while testing various compatibility scenarios.

Here the important bits in tox.ini

[tox]
minversion = 3.7
envlist = py{3}-ansible{latest,29,28}-{   alpinelatest,alpine310,alpine39,alpine38,   centoslatest,centos8,centos7,   debianlatest,debian10,debian9,debian8,   fedoralatest,fedora30,fedora29,fedora28,   ubuntulatest,ubuntu2004,ubuntu1904,ubuntu1804,ubuntu1604   }

# only test currently supported ansible versions
# https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#release-status

skipsdist = true

[base]
passenv = *
deps =
    -rrequirements.txt
    ansible25: ansible==2.5
    ...
    ansiblelatest: ansible
commands =
    molecule test
setenv =
    TOX_ENVNAME={envname}
    MOLECULE_EPHEMERAL_DIRECTORY=/tmp/{envname}

[testenv]
passenv = 
    {[base]passenv}
deps = 
    {[base]deps}
commands = 
    {[base]commands}
setenv =
    ...
    centoslatest:     MOLECULE_DISTRO="centos:latest"
    centos8:          MOLECULE_DISTRO="centos:8"
    centos7:          MOLECULE_DISTRO="centos:7"
    ...
    {[base]setenv}

The entirety of requirements.txt

docker
molecule

by simply executing

tox

it will create virtual envs for each compatibility combination defined in tox.ini by

envlist = py{3}-ansible{latest,29,28}-{   alpinelatest,alpine310,alpine39,alpine38,   centoslatest,centos8,centos7,   debianlatest,debian10,debian9,debian8,   fedoralatest,fedora30,fedora29,fedora28,   ubuntulatest,ubuntu2004,ubuntu1904,ubuntu1804,ubuntu1604   }

which translates in this particular case to: python3 x ansible version x linux distro

Great! We have created tests for compatibility checks with the added benefit of always testing with ansible latest to notice breaking changes early on.

The files used in this step

• tox.ini https://github.com/ckaserer/ansible-role-example/blob/master/tox.ini
• requirements.txt https://github.com/ckaserer/ansible-role-example/blob/master/requirements.txt

Source

• https://tox.readthedocs.io/en/latest/
• https://molecule.readthedocs.io/en/stable/testing.html#tox

---

Step 3: CI with travis

Running the checks locally is good, running in a CI tool is great. So let's do that.

For this purpose following bits in the .travis.yml are important

---
version: ~> 1.0

os: linux
language: python

python:
  - "3.8"
  - "3.7"
  - "3.6"
  - "3.5"

services: docker

cache:
  pip: true
  directories:
  - .tox

install:
  - pip install tox-travis
env:
  jobs:
    # ansible:latest - check for breaking changes
    ...
    - TOX_DISTRO="centoslatest"     TOX_ANSIBLE="latest"
    - TOX_DISTRO="centos8"          TOX_ANSIBLE="latest"
    - TOX_DISTRO="centos7"          TOX_ANSIBLE="latest"
    ...
    # ansible: check version compatibility
    # only test currently supported ansible versions
    # 
https://docs.ansible.com/ansible/latest/reference_appendices/release_and_maintenance.html#release-status
    - TOX_DISTRO="centoslatest"     TOX_ANSIBLE="{29,28}"
    - TOX_DISTRO="centos8"          TOX_ANSIBLE="{29,28}"
    - TOX_DISTRO="centos7"          TOX_ANSIBLE="{29,28}"
    ...  
script:
  - tox -e $(echo py${TRAVIS_PYTHON_VERSION} | tr -d .)-ansible${TOX_ANSIBLE}-${TOX_DISTRO}
   # remove logs/pycache before caching .tox folder  
  - |
    rm -r .tox/py*/log/*                                                               
    find . -type f -name '*.py[co]' -delete -o -type d -name __pycache__ -delete

First we have specified language: python to run builds with multiple versions of python as defined in the python: list.

We need docker, so we add it via services: docker.

The test will take quite some time, let's cache pip and our virtenv created by tox with

cache:
  pip: true
  directories:
  - .tox

We need tox...

install:
  - pip install tox-travis

And finally, we define all our test cases

env:
  jobs:
    # ansible:latest - check for breaking changes
    ...
    - TOX_DISTRO="centoslatest"     TOX_ANSIBLE="latest"
    ...

Note that I have separate jobs for latest and the distinct versions. That is on purpose. I would like to easily see what broke. Is it version compatibility or an upcoming change located in ansible's latest release.

The files used in this step

• https://github.com/ckaserer/ansible-role-example/blob/master/.travis.yml

Source

• https://docs.travis-ci.com/user/caching/

---

Bonus: run tox in parallel

You can run the tests in parallel (e.g. 3 test simultaneous) by executing

tox -p 3

However, this will not give the output from molecule. You can enable that with

tox -p 3 -o true

The obvious downside to this approach is the pain of figuring out which line belongs to which process in the parallel execution.

Source

• https://tox.readthedocs.io/en/latest/example/basic.html#parallel-mode

Answer 2

No real answer here, but some ideas :

Ansible Silo might have fitted, but no commit for a year.

And it's not exactly what you're looking for, but Ansible Runner is meant to be a fit for the "run ansible" use case. And it's a part of Ansible Tower / AWS, so it should last.

Runner is intended to be most useful as part of automation and tooling that needs to invoke Ansible and consume its results.

They do mention executing from a container here

The design of Ansible Runner makes it especially suitable for controlling the execution of Ansible from within a container for single-purpose automation workflows

But the issue for you is that the official ansible/ansible-runner container is tagged after ansible-runner version, and ansible itself is installed through pip install ansible at container build time.