Remove duplicate lines from Ansible variable

Question

I need to achieve two things.

1. Remove duplicate entries from ansible variable called install_loc

2. Remove any empty blank lines from install_loc.

Below is how install_loc variable is constructed.

   - set_fact:
       install_loc: "{{ install_loc | default('') + item.split( )[4] | dirname + '\n' }}"
     loop: "{{ fdet }}"

Below are the contents of install_loc after writing it to a file.

/app/logs/scripts
/app/logs/mrt
/app/logs/com
/app/logs/exe

/app/logs/scripts
/app/logs/mrt
/app/logs/com
/app/logs/exe

/app/logs/scripts
/app/logs/mrt
/app/logs/com
/app/logs/exe

As you can see the variable as newlines as well as duplicate entries.

Desired output should be as below. Note: Order does not matter:

/app/logs/scripts
/app/logs/mrt
/app/logs/com
/app/logs/exe

I tried

   - set_fact:
       install_loc: "{{ install_loc | unique }}"  

But i get distorted text like below:

set([u'\n', u'/', u'.', u'1', u'0', u'3', u'2', u'C', u'E', u'G', u'F', u'I', u'N', u'a', u'c', u'e', u't', u'i', u'm', u'o', u'o', u'n', u's', u's', u'r', u'u', u't', u'x'])

Can you please suggest ?

Answer 1

The solution:

- set_fact:
    install_loc: "{{ install_loc.split('\n') | unique | select | list }}"

Explanation:

• install_loc.split('\n') - Split the install_loc by newlines
• unique - Remove duplicates
• select - Get rid of empty or null values
• list - Convert from a Python "generator" to a list

If you want a single string (as the input) replace list by join('\n').