Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When to use from_json filter in Ansible?

Tags:

ansible

When should I use the from_json filter in Ansible?

I found out that using it sometimes has and sometimes have no effect.

Please consider the following example which illustrates the inconsistency I am getting.

Included in reverse order are: the questions - expected result - actual result - the playbook - the data. The data is taken from this question and the playbook is based on this answer.

  • The question(s):

    Why storing the left part (before json_query) of the following expression in a variable and then using json_query on the variable causes the expression to be evaluated differently?

    "{{ lookup('file','test.json') | json_query(query) }}"
    

    Why does adding from_json filter alter the results (but does not if processing a variable):

    "{{ lookup('file','test.json') | from_json | json_query(query) }}"
    
  • Expected result:

    Last four tasks should give the same result. Alternatively, last two tasks should give the same result as previous two tasks.

  • Actual result (last four tasks only):

    One task result differs.

    TASK [This query is run against lookup value with from_json stored in a variable] ***
    ok: [localhost] => {
        "msg": [
            678
        ]
    }
    
    TASK [This query is run against lookup value without from_json stored in a variable] ***
    ok: [localhost] => {
        "msg": [
            678
        ]
    }
    
    TASK [This query is run directly against lookup value with from_json] **********
    ok: [localhost] => {
        "msg": [
            678
        ]
    }
    
    TASK [This query is run directly against lookup value without from_json - the result is empty - why?] ***
    ok: [localhost] => {
        "msg": ""
    }
    
  • The playbook:

    ---
    - hosts: localhost
      gather_facts: no
      connection: local
      tasks:   
        - set_fact:
            from_lookup_with_from_json: "{{ lookup('file','test.json') | from_json }}"
    
        - set_fact:
            from_lookup_without_from_json: "{{ lookup('file','test.json') }}"
    
        - name: Save the lookup value stored in a variable in a file for comparison
          copy: content="{{ from_lookup_with_from_json }}" dest=./from_lookup_with_from_json.txt
    
        - name: Save the lookup value stored in a variable in a file for comparison (they are the same)
          copy: content="{{ from_lookup_without_from_json }}" dest=./from_lookup_without_from_json.txt
    
        - name: This query is run against lookup value with from_json stored in a variable
          debug: msg="{{ from_lookup_with_from_json | json_query(query) }}"
          vars:
            query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
    
        - name: This query is run against lookup value without from_json stored in a variable
          debug: msg="{{ from_lookup_without_from_json | json_query(query) }}"
          vars:
            query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
    
        - name: This query is run directly against lookup value with from_json
          debug: msg="{{ lookup('file','test.json') | from_json | json_query(query) }}"
          vars:
            query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
    
        - name: This query is run directly against lookup value without from_json - the result is empty - why?
          debug: msg="{{ lookup('file','test.json') | json_query(query) }}"
          vars:
            query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
    
  • The data (test.json):

    { "Foods" :
      [ { "Id": 456
        , "Tags":
          [ {"Key":"For", "Value":"Heffalump"}
          , {"Key":"Purpose", "Value":"Food"}
          ]
        }
      , { "Id": 678
        , "Tags":
          [ {"Key":"For", "Value":"Tigger"}
          , {"Key":"Purpose", "Value":"Food"}
          ]
        }
      , { "Id": 911
        , "Tags":
          [ {"Key":"For", "Value":"Roo"}
          , {"Key":"Purpose", "Value":"Food"}
          ]
        }
      ]
    }
    
like image 313
techraf Avatar asked Dec 21 '16 13:12

techraf


1 Answers

json_query requires Python object (dict) as input, if you feed it with string, it gives empty string as result.

You get different result because of Ansible templating engine tricky work.
I should definitely write a post about it on my site...

After evaluating jijna2 expression Ansible try to cast complex types to Python objects (like dict or list). See my other answer.

In your case:

1.

- set_fact:
    from_lookup_with_from_json: "{{ lookup('file','test.json') | from_json }}"

from_lookup_with_from_json is a dict, because you manually convert JSON-string from file to dict with from_json filter.

2.

- set_fact:
    from_lookup_without_from_json: "{{ lookup('file','test.json') }}"

from_lookup_with_from_json becomes dict, because Ansible converts it when jinja2 expression ends with }}. So from_json is actually unnecessary as the last filter in chain.

3.

  debug: msg="{{ lookup('file','test.json') | from_json | json_query(query) }}"

Again, you manually convert JSON-string here. So json_query get dict as input.

4.

  debug: msg="{{ lookup('file','test.json') | json_query(query) }}"

In this case you feed JSON-string (not dict) as input to json_query filter. As everything happens inside one jinja2 expression, Ansible doesn't attempt to convert anything in between.

You can also get empty string result with a variable this way:

- set_fact:
    from_lookup_force_string: "{{ lookup('file','test.json') | string }}"

In this case from_lookup_force_string will not be converted by Ansible tempating engine, and json_query will give you empty response on it.

like image 145
Konstantin Suvorov Avatar answered Sep 20 '22 08:09

Konstantin Suvorov