Ansible: How to optimize running speed of a playbook?

Question

Current Setup

CPU/RAM 60c/64GB ~2000 servers Time consuming ~1 hour

Ansible playbook actions

• Create folders for each server on localhost and filling them up with templates (vars taken from host inventory's)
• Encrypting the folders
• and all this folders are pushed to 2 Remote Servers

The most time consuming step is the one which creates the folders for each config file. And so on for 2k servers.

What I have tried

• Increasing from 50 > 100 > 400> 800 forks

  • There was improvement in about 5% no more.

• Switching to free strategy takes even more time that linear.

Any ideas how to make it at least x2 faster?

The heaviest task

- name: Create blablabla
  file:
    path: "{{ item.value }}"
    state: directory
    mode: "{{ item.mode | default('0755') }}"
  with_items:
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX/ZZZZ.d", create: "{{ not WWWW }}" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX", create: "{{ not WWWW }}" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX/EEEE/QQQQ-versions.d" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX.d/SSS" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX.d/EEE/WEWEW" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX/EEE" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXXX/WWW" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXXX/DD" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXXX/SSS/DDD-QQQQ.d" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXXX/DD" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXXX/DDD/conf" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX/DDD/.docker" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX/DD/gpg", mode: "0700" }
    - { value: "{{ some_dir }}/{{ inventory_hostname }}/XXX/DDD.gpg.d", mode: "0755" }

Answer 1

Most time consuming step is the one which creates the folders for each config file.

Right. This is the expected behavior since the file module can't work with a list for parameter path and therefore has to create a single connection for each parameter. This is causing a significant overhead.

Any ideas how to make it at least x2 faster?

I understand that you like to know "How to increase the overall playbook performance and decrease the execution time of specific tasks?" and furthermore you do not want to loop over the directories, instead providing a list of directories.

Yes. Since your problem is quite common and other users experience the same, including myself, I've wrote up a possible solution approach under Ansible - with_items pointed directly at vars rather than expanding them first. It shows specifically the test results and differences in execution time.

Further Similiar Q&A are

• How do I optimize performance of Ansible playbook with regards to SSH connections?
• Parallel execution of localhost tasks in Ansible?

As one can see from the results it can become easily 4 times faster and even much more. It also shows why using async can make the issue worst under certain circumstances.

To summarize, the simplest, quick and lazy approach can be just to use the command or shell module with cmd: mkdir -p ... and as shown here already.

---

Another approach and depending on your infrastructure and other capabilities could be Creating an own Custom Module in Bash or Shell.

Examples can be find under

• File search between two dates with Ansible
• How to execute a shell script on a Remote Server using Ansible?

In general it would just take a list of directories in one string as path parameter

path: "/this,/is,/my,/list,/of,/directories,/to,/create"

and perform a kind of

mkdir -p {$path}

within the module.