Multistage deployment with ansible

Question

What approach would you advise to organize multistage deployment with Ansible in case you have different variables for stages?

The main idea is defining group variables for different stages.

There are two articles:

• http://rosstuck.com/multistage-environments-with-ansible/
• http://toja.io/using-host-and-group-vars-files-in-ansible/

I'd like to get more examples about organizing playbooks, variables, and hosts, and understand advantages and disadvantages of your approach.

Answer 1

Recently, I used the approach I had already mentioned in the question, and it occurred to be one of the most convenient to my mind.

It is taken from Organizing Group Vars Files in Ansible article, but altered a little, because, unfortunately, title of the article does not reflect it's real value and purpose and names of playbooks are confusing as well. In fact, it took a considerable time to realize that it is about Multistage deployment with Ansible.

Your layout of directories should be like that:

production/
├── group_vars
│   └── server.yml
└── inventory
staging/
├── group_vars
│   └── server.yml
└── inventory
deploy.yml

And usage is extremely simple:

ansible-playbook -i staging deploy.yml

Where deploy.yml is the name of your playbook.

Ansible-playbook when provided a directory as the inventory, will search by default a file named inventory so no need to specify -i production/inventory, only -i production will work just fine.

And the benefits are:

• You don't have to mantain some unnecessary groups like [production:children]

• You don't need to keep confusing groups and files like like group_vars/production.yml

• All vars and hosts are in separate directories, therefore it is easy to keep them different and history of changes is clear. You may even split it in separate repositories if you want

You may also keep secrets for production in repository using ansible-vault, in other words, store all your vital variables encrypted

Answer 2

In case of complex inventory structures, when maintaining groups of groups is not the best option ( http://docs.ansible.com/ansible/intro_inventory.html#groups-of-groups-and-group-variables ) the following trick can be used:

Production Inventory File:

# production inventory 
[loadbalancers]
lb01
lb02
lb03

[webservers]
ws01
ws02
ws03

[all:vars]
inventory_vars=prod.config.yml

Development Inventory File:

# development inventory 
[loadbalancers]
test-lb01
test-lb02
test-lb03

[webservers]
test-ws01
test-ws02
test-ws03

[all:vars]
inventory_vars=development.config.yml

Now in playbook itself, include the following task before loading roles:

- hosts: all

  pre_tasks:

    - name: Load inventory specific variables
      include_vars: "{{ inventory_vars }}"

To prevent accidental execution of playbook on production environment, prod.config.yml cab be encrypted with ansible-vault

Answer 3

Currently I'm using the following structure:

hosts/development
hosts/production
hosts/group_vars/development/service1.yml
hosts/group_vars/development/service2.yml
hosts/group_vars/production/service1.yml
hosts/group_vars/production/service2.yml
hosts/group_vars/production/service3.yml
hosts/host_vars/dev1.yml
hosts/host_vars/prod1/something.yml
hosts/host_vars/prod1/something_else.yml

The inventories could look like this:

# hosts/development

dev1 ansible_ssh_host=dev1.example.com
dev2 ansible_ssh_host=dev2.example.com

[development]
dev1
dev2

[service1]
dev1

[service2]
dev2

[service3]
dev1
dev2

And for production:

# hosts/production

prod1 ansible_ssh_host=prod1.example.com
prod2 ansible_ssh_host=prod2.example.com

[production]
prod1
prod2

[service1]
prod1

[service2]
prod2

[service3]
prod1
prod2

This allows for some nice combinations. By using ansible -i hosts I can target all known hosts. I'm using this, for example, to add all servers in the inventory to a monitoring configuration file.

By using ansible -i hosts/development I can limit the command to the development (or production) servers. I do this when I want to test a new configuration on the development system before I apply it to production.

I'm currently using this structure for about 25 servers on 3 different stages and it works pretty well for me. It has some weaknesses, though:

• Although in my case the advantages overweigh those weaknesses. My biggest fear is accidentally targeting all inventories in the hosts directory, by forgetting to limit to one of the stages. That would be easier to avoid if the inventories would be completely separate.
• Also, I don't like having to list all servers in the inventory again in the development or production groups, because it's redundant and easy to forget.
• I guess if your system grows it could become a little bit confusing to understand from where and in which order variables are loaded.

That being said, it works pretty well for me, so maybe it works well for you, too.