Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

debugging with ansible: How to get stderr and stdout from failing commands to be printed respecting newlines so as to be human readable?

Tags:

ansible

If I run a simple ansible playbook, I often get difficult-to-read output from failing tasks, like that below. Big problems:

  • the linebreaks within the stdout are printed as \n, not an actual linebreak. This makes things like python tracebacks very obnoxious to read.

  • stdout, stderr, cmd... the json blob being output contains lots of useful things, but since they are all run together on the same line it is very difficult for a human to parse.

How can I get ansible to print its output in a format that I can read easily, so I can debug?

Here is the yucky output:

$ ansible-playbook playbooks/backUpWebsite.yml

PLAY [localhost] 
***************************************************************

TASK [command] 
*****************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "python -c 'ksjfasdlkjf'", "delta": "0:00:00.037459", "end": "2017-10-03 19:58:50.525257", "failed": true, "rc": 1, "start": "2017-10-03 19:58:50.487798", "stderr": "Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nNameError: name 'ksjfasdlkjf' is not defined", "stdout": "", "stdout_lines": [], "warnings": []}
        to retry, use: --limit @<snip>playbooks/backUpWebsite.retry

PLAY RECAP 
*********************************************************************
localhost                  : ok=0    changed=0    unreachable=0 failed=1   

Here is the script that generated it:

---
- hosts: localhost
  gather_facts: False
  tasks:
      #wrong on purpose!
    - shell: "python -c 'ksjfasdlkjf'"
      register: unobtainable
like image 585
stochastic Avatar asked Oct 04 '17 02:10

stochastic


People also ask

How do you debug Ansible commands?

You can enable the task debugger globally with a setting in ansible. cfg or with an environment variable. The only options are True or False . If you set the configuration option or environment variable to True , Ansible runs the debugger on failed tasks by default.

How do I debug an Ansible module?

Extract the module you want to debug from the zipped file that Ansible sent to the remote host: $ python AnsiballZ_my_test_module.py explode . Ansible will expand the module into ./debug-dir . You can optionally run the zipped file by specifying python AnsiballZ_my_test_module.py .

How do you capture command output in Ansible?

To capture the output, you need to specify your own variable into which the output will be saved. To achieve this, we use the 'register' parameter to record the output to a variable. Then use the 'debug' module to display the variable's content to standard out.

Why debug module gets used in Ansible?

This module prints statements during execution and can be useful for debugging variables or expressions without necessarily halting the playbook. Useful for debugging together with the 'when:' directive. This module is also supported for Windows targets.


1 Answers

To get ansible to respect linebreaks in output so that you can read the stdout of your failing task, change the stdout_callback to minimal.

In your ansible.cfg, change the stdout_callback option:

stdout_callback=minimal

the 'minimal' output is a bit terse, but it prints output respecting linebreaks so that you can see what is happening:

 $ ansible-playbook playbooks/backUpWebsite.yml
 localhost | FAILED | rc=1 >>
 Traceback (most recent call last):
   File "<string>", line 1, in <module>
 NameError: name 'ksjfasdlkjf' is not defined

         to retry, use: --limit @<snip>/playbooks/backUpWebsite.retry

Other things that I hear can work

  • You could write your own callback handler for stdout. People have done this and it seems straightforward to do, but I've never tried it.

Things I tried that don't work:

Three vs. Use "ansible-playbook -vvv". You get a tremendous pile of output, which does include the json nicely linebroken, but the newlines within the stdout output are not printed, so stdout/stderr are still hard to read:

$ ansible-playbook playbooks/backUpWebsite.yml -vvv
Using <snip>/ansible.cfg as config file

PLAYBOOK: backUpWebsite.yml ****************************************************
1 plays in playbooks/backUpWebsite.yml

PLAY [localhost] ***************************************************************

TASK [command] *****************************************************************
task path: <snip>/playbooks/backUpWebsite.yml:6
Using module file <snip>/ansible/lib/ansible/modules/core/commands/command.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: <snip>
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205 `" && echo ansible-tmp-1507083519.4247632-54588405727205="` echo <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmpumtlourx TO <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/command.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/ <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/command.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '<snip>/anaconda3/bin/python <snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/command.py; rm -rf "<snip>/.ansible/tmp/ansible-tmp-1507083519.4247632-54588405727205/" > /dev/null 2>&1 && sleep 0'
fatal: [localhost]: FAILED! => {
    "changed": true,
    "cmd": "python -c 'ksjfasdlkjf'",
    "delta": "0:00:00.037657",
    "end": "2017-10-03 20:18:39.843933",
    "failed": true,
    "invocation": {
        "module_args": {
            "_raw_params": "python -c 'ksjfasdlkjf'",
            "_uses_shell": true,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "warn": true
        },
        "module_name": "command"
    },
    "rc": 1,
    "start": "2017-10-03 20:18:39.806276",
    "stderr": "Traceback (most recent call last):\n  File \"<string>\", line 1, in <module>\nNameError: name 'ksjfasdlkjf' is not defined",
    "stdout": "",
    "stdout_lines": [],
    "warnings": []
}
        to retry, use: --limit @<snip>/sys/ansible_readynet/playbooks/backUpWebsite.retry

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1   

Credit

I posted this issue against ansible on github in hope of persuading people that the ansible defaults are very unhelpful. jhawkesworth helpfully introduced me to the 'minimal' stdout callback.

I have since found this page in the ansible documentation that mentions this plugin in a very non-prominent way.

IMHO, this secret of getting readable error output from ansible is entirely too well hidden from new users, and I have personally witnessed this sour several people I know on the use of ansible in general.

like image 197
stochastic Avatar answered Sep 18 '22 14:09

stochastic