Is there a way to use doctest and sphinx to test and document command line applications?

Question

I have a Python module for which I'm writing a tutorial using Sphinx including doctests.

This module comes with a few helper programs.

I would like to include those helper programs in the documentation and have doctest check that the standard output is in sync between the current program version and the documentation.

I suppose I can use the sh module, or popen to check the standard output of a given program but I prefer that those tricks do not show up into the docs, or else non-programmers users will be certainly lost.

Is there a way to achieve that?

Answer 1

The doctest module only checks statements than can be run from the python interactive prompt.

Command-line tools can be invoked from the python interactive prompt using the subprocess module:

# Create Helper Function
>>> import subprocess
>>> run_commandline = lambda cmd: subprocess.check_output(cmd, shell=True).decode()

# Doctestable command-line calls
>>> print(run_commandline('cal 7 2017'))
     July 2017
Su Mo Tu We Th Fr Sa
                   1
 2  3  4  5  6  7  8
 9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31

>>> print(run_commandline('echo $BASH_VERSION'))
3.2.57(1)-release

There are likely some ways to hack doctest or sphinx to get what you want more directly, but this technique uses the advertised APIs for doctest, sphinx, and subprocess in exactly the ways they were designed to be used (doctest is designed to replay interactive prompt sessions found in docstrings, and subprocess is designed to run command-line tools directly from python and capture their output).

I suppose I can use the sh module, or popen to check the standard output of a given program but I prefer that those tricks do not show up into the docs, or else non-programmers users will be certainly lost.

Two thoughts: First, the details of those calls can mostly be hidden in a helper function to minimize distraction. Second, if you need to invoke command-line programs from Python, it isn't a trick to use popen or subprocess since those are the tools designed specifically for making those calls from Python.