ansible - delete unmanaged files from directory?

Question

I want to recursively copy over a directory and render all .j2 files in there as templates. For this I am currently using the following lines:

- template: >
            src=/src/conf.d/{{ item }}
            dest=/dest/conf.d/{{ item|replace('.j2','') }}
  with_lines: find /src/conf.d/ -type f -printf "%P\n"

Now I'm looking for a way to remove unmanaged files from this directory. For example if I remove a file/template from /src/conf.d/ I want Ansible to remove it from /dest/conf.d/ as well.

Is there some way to do this? I tried fiddling around with rsync --delete, but there I got a problem with the templates which get their suffix .j2 removed.

Answer 1

I'd do it like this, assuming a variable defined as 'managed_files' up top that is a list.

- shell: ls -1 /some/dir
  register: contents

- file: path=/some/dir/{{ item }} state=absent
  with_items: contents.stdout_lines
  when: item not in managed_files

Answer 2

We do this with our nginx files, since we want them to be in a special order, come from templates, but remove unmanaged ones this works:

# loop through the nginx sites array and create a conf for each file in order 
# file will be name 01_file.conf, 02_file.conf etc
- name: nginx_sites conf
  template: >
    src=templates/nginx/{{ item.1.template }}
    dest={{ nginx_conf_dir }}/{{ '%02d' % item.0 }}_{{ item.1.conf_name|default(item.1.template) }}
    owner={{ user }}
    group={{ group }}
    mode=0660
  with_indexed_items: nginx_sites
  notify:
    - restart nginx 
  register: nginx_sites_confs

# flatten and map the results into simple list
# unchanged files have attribute dest, changed have attribute path
- set_fact:
    nginx_confs: "{{ nginx_sites_confs.results|selectattr('dest', 'string')|map(attribute='dest')|list + nginx_sites_confs.results|selectattr('path', 'string')|map(attribute='path')|select|list }}"
  when: nginx_sites

# get contents of conf dir
- shell: ls -1 {{ nginx_conf_dir }}/*.conf
  register: contents
  when: nginx_sites

# so we can delete the ones we don't manage
- name: empty old confs 
  file: path="{{ item }}" state=absent
  with_items: contents.stdout_lines
  when: nginx_sites and item not in nginx_confs

The trick (as you can see) is that template and with_items have different attributes in the register results. Then you turn them into a list of files you manage and then get a list of the the directory and removed the ones not in that list.

Could be done with less code if you already have a list of files. But in this case I'm creating an indexed list so need to create the list as well with map.

Answer 3

I want to share my experience with this case.

Ansible from 2.2 is had with_filetree loop provides simple way to upload dirs, links, static files and even (!) templates. It's best way to keep my config dir synchronized.

- name: etc config - Create directories
  file:
    path: "{{ nginx_conf_dir }}/{{ item.path }}"
    state: directory
    mode: 0755
  with_filetree: etc/nginx
  when: item.state == 'directory'

- name: etc config - Creating configuration files from templates
  template:
    src: "{{ item.src }}"
    dest: "{{ nginx_conf_dir }}/{{ item.path | regex_replace('\\.j2$', '') }}"
    mode: 0644
  with_filetree: etc/nginx
  when:
    - item.state == "file"
    - item.path | match('.+\.j2$') | bool

- name: etc config - Creating staic configuration files
  copy:
    src: "{{ item.src }}"
    dest: "{{ nginx_conf_dir }}/{{ item.path }}"
    mode: 0644
  with_filetree: etc/nginx
  when:
    - item.state == "file"
    - not (item.path | match('.+\.j2$') | bool)

- name: etc config - Recreate symlinks
  file:
    src: "{{ item.src }}"
    dest: "{{ nginx_conf_dir }}/{{ item.path }}"
    state: link
    force: yes
    mode: "{{ item.mode }}"
  with_filetree: etc/nginx
  when: item.state == "link"

Next we may want delete unused files from config dir. It's simple. We gather list of uploaded files and files exist on remote server, next remove diffrence.

But we may want to have unmanaged files in config dir. I've used -prune functionality of find to avoid clearing folders with unmanaged files.

PS _(Y)_ sure after I have deleted some unmanaged files

- name: etc config - Gathering managed files
  set_fact:
    __managed_file_path: "{{ nginx_conf_dir }}/{{ item.path | regex_replace('\\.j2$', '') }}"
  with_filetree: etc/nginx
  register: __managed_files

- name: etc config - Convert managed files to list
  set_fact: managed_files="{{ __managed_files.results | map(attribute='ansible_facts.__managed_file_path') | list }}"

- name: etc config - Gathering exist files (excluding .ansible_keep-content dirs)
  shell: find /etc/nginx -mindepth 1 -type d -exec test -e '{}/.ansible_keep-content' \; -prune -o -print
  register: exist_files
  changed_when: False

- name: etc config - Delete unmanaged files
  file: path="{{ item }}" state=absent
  with_items: "{{ exist_files.stdout_lines }}"
  when:
    - item not in managed_files