Pytest with argparse: how to test user is prompted for confirmation?

Question

I have a CLI tool, and would like to test that the user is prompted to confirm a choice using input(). This would be equivalent as using raw_input() in Python 2.

Code

The (paraphrased) code to test looks like:

import sys
import argparse


def confirm():
    notification_str = "Please respond with 'y' or 'n'"
    while True:
        choice = input("Confirm [Y/n]?").lower()
        if choice in 'yes' or not choice:
            return True
        if choice in 'no':
            return False
        print(notification_str)


def parse_args(args):
    parser = argparse.ArgumentParser()
    parser.add_argument('-d', '--destructive', action='store_true')
    return parser.parse_args()


def main():
    args = parse_args(sys.argv[1:])
    if args.destructive:
        if not confirm():
            sys.exit()
    do_stuff(args)


if __name__ == '__main__':
    main()

Question

I am using pytest as my framework. How do I make it so I can test that the confirmation prompt is showing up in the CLI? If I try to compare stdout I get the error: OSError: reading from stdin while output is captured.

I want to make sure that:

1. The confirmation shows up when the destructive flag is set
2. It doesn't show up when it isn't

I will be using the following code in another file:

import pytest
from module_name import main


def test_user_is_prompted_when_destructive_flag_is_set():
    sys.argv['', '-d']
    main()
    assert _  # What the hell goes here?


def test_user_is_not_prompted_when_destructive_flag_not_set():
    sys.argv['',]
    main()
    assert _  # And here too?

Answer 1

I would suggest that starting testing with the confirm() function is a better unit test strategy. This allows things like input and sys.stdio to be mocked more locally. Then once assured confirms works as expected, tests can be written that verify that it is called in specific ways. You can write tests for that, and mock confirm() during those tests.

Here is a unit test forconfirm() that uses pytest.parametrize and mock to deal with user input and output:

Code:

@pytest.mark.parametrize("from_user, response, output", [
    (['x', 'x', 'No'], False, "Please respond with 'y' or 'n'\n" * 2),
    ('y', True, ''),
    ('n', False, ''),
    (['x', 'y'], True, "Please respond with 'y' or 'n'\n"),
])
def test_get_from_user(from_user, response, output):
    from_user = list(from_user) if isinstance(from_user, list) else [from_user]

    with mock.patch.object(builtins, 'input', lambda x: from_user.pop(0)):
        with mock.patch('sys.stdout', new_callable=StringIO):
            assert response == confirm()
            assert output == sys.stdout.getvalue()

How does this work?

pytest.mark.parametrize allows a test function to be easily called multple times with conditions. Here are 4 simple steps which will test most of the functionality in confirm:

@pytest.mark.parametrize("from_user, response, output", [
    (['x', 'x', 'No'], False, "Please respond with 'y' or 'n'\n" * 2),
    ('y', True, ''),
    ('n', False, ''),
    (['x', 'y'], True, "Please respond with 'y' or 'n'\n"),
])

mock.patch can be used to temporarily replace a function in module (among other uses). In this case it is used to replace input and sys.stdout to allow inject user input, and capture printed strings

with mock.patch.object(builtins, 'input', lambda x: from_user.pop(0)):
    with mock.patch('sys.stdout', new_callable=StringIO):

finally the function under test is run and the output of the function and any string printed are verified:

assert response == confirm()
assert output == sys.stdout.getvalue()

Test Code (for the test code):

import sys
from io import StringIO
import pytest
from unittest import mock
import builtins

def confirm():
    notification_str = "Please respond with 'y' or 'n'"
    while True:
        choice = input("Confirm [Y/n]?").lower()
        if choice in 'yes' or not choice:
            return True
        if choice in 'no':
            return False
        print(notification_str)

@pytest.mark.parametrize("from_user, response, output", [
    (['x', 'x', 'No'], False, "Please respond with 'y' or 'n'\n" * 2),
    ('y', True, ''),
    ('n', False, ''),
    (['x', 'y'], True, "Please respond with 'y' or 'n'\n"),
])
def test_get_from_user(from_user, response, output):
    from_user = list(from_user) if isinstance(from_user, list) \
        else [from_user]
    with mock.patch.object(builtins, 'input', lambda x: from_user.pop(0)):
        with mock.patch('sys.stdout', new_callable=StringIO):
            assert response == confirm()
            assert output == sys.stdout.getvalue()

pytest.main('-x test.py'.split())

Results:

============================= test session starts =============================
platform win32 -- Python 3.6.3, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: C:\Users\stephen\Documents\src\testcode, inifile:
collected 4 items

test.py ....                                                             [100%]

========================== 4 passed in 0.15 seconds ===========================

Test Calls to confirm():

To test that confirm is called when expected, and that the program responds as expected when called, you can use unittest.mock to mock confirm().

Note: In the usual unittest scenario, confirm would be in a different file and mock.patch could be used in a similiar manner to how sys.argv is patched in this example.

Test Code for checking calls to confirm():

import sys
import argparse

def confirm():
    pass

def parse_args(args):
    parser = argparse.ArgumentParser()
    parser.add_argument('-d', '--destructive', action='store_true')
    return parser.parse_args()


def main():
    args = parse_args(sys.argv[1:])
    if args.destructive:
        if not confirm():
            sys.exit()


import pytest
from unittest import mock

@pytest.mark.parametrize("argv, called, response", [
    ([], False, None),
    (['-d'], True, False),
    (['-d'], True, True),
])
def test_get_from_user(argv, called, response):
    global confirm
    original_confirm = confirm
    confirm = mock.Mock(return_value=response)
    with mock.patch('sys.argv', [''] + argv):
        if called and not response:
            with pytest.raises(SystemExit):
                main()
        else:
            main()

        assert confirm.called == called
    confirm = original_confirm

pytest.main('-x test.py'.split())

Results:

============================= test session starts =============================
platform win32 -- Python 3.6.3, pytest-3.3.2, py-1.5.2, pluggy-0.6.0
rootdir: C:\Users\stephen\Documents\src\testcode, inifile:
collected 3 items

test.py ...                                                              [100%]

========================== 3 passed in 3.26 seconds ===========================
enter code here