Python CLI program unit testing

Question

I am working on a python Command-Line-Interface program, and I find it boring when doing testings, for example, here is the help information of the program:

usage: pyconv [-h] [-f ENCODING] [-t ENCODING] [-o file_path] file_path  Convert text file from one encoding to another.  positional arguments:   file_path  optional arguments:   -h, --help            show this help message and exit   -f ENCODING, --from ENCODING                         Encoding of source file   -t ENCODING, --to ENCODING                         Encoding you want   -o file_path, --output file_path                         Output file path 

When I made changes on the program and want to test something, I must open a terminal, type the command(with options and arguments), type enter, and see if any error occurs while running. If error really occurs, I must go back to the editor and check the code from top to end, guessing where the bug positions, make small changes, write print lines, return to the terminal, run command again...

Recursively.

So my question is, what is the best way to do testing with CLI program, can it be as easy as unit testing with normal python scripts?

Answer 1

I think it's perfectly fine to test functionally on a whole-program level. It's still possible to test one aspect/option per test. This way you can be sure that the program really works as a whole. Writing unit-tests usually means that you get to execute your tests quicker and that failures are usually easier to interpret/understand. But unit-tests are typically more tied to the program structure, requiring more refactoring effort when you internally change things.

Anyway, using py.test, here is a little example for testing a latin1 to utf8 conversion for pyconv::

# content of test_pyconv.py  import pytest  # we reuse a bit of pytest's own testing machinery, this should eventually come # from a separatedly installable pytest-cli plugin.  pytest_plugins = ["pytester"]  @pytest.fixture def run(testdir):     def do_run(*args):         args = ["pyconv"] + list(args)         return testdir._run(*args)     return do_run  def test_pyconv_latin1_to_utf8(tmpdir, run):     input = tmpdir.join("example.txt")     content = unicode("\xc3\xa4\xc3\xb6", "latin1")     with input.open("wb") as f:         f.write(content.encode("latin1"))     output = tmpdir.join("example.txt.utf8")     result = run("-flatin1", "-tutf8", input, "-o", output)     assert result.ret == 0     with output.open("rb") as f:         newcontent = f.read()     assert content.encode("utf8") == newcontent 

After installing pytest ("pip install pytest") you can run it like this::

$ py.test test_pyconv.py =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- pytest-2.4.5dev1 collected 1 items  test_pyconv.py .  ========================= 1 passed in 0.40 seconds ========================= 

The example reuses some internal machinery of pytest's own testing by leveraging pytest's fixture mechanism, see http://pytest.org/latest/fixture.html. If you forget about the details for a moment, you can just work from the fact that "run" and "tmpdir" are provided for helping you to prepare and run tests. If you want to play, you can try to insert a failing assert-statement or simply "assert 0" and then look at the traceback or issue "py.test --pdb" to enter a python prompt.