How to use Python to programmatically generate part of Sphinx documentation?

Question

I am using Sphinx to generate the documentation for a project of mine.

In this project, I describe a list of available commands in a yaml file which, once loaded, results in a dictionary in the form {command-name : command-description} for example:

commands = {"copy"  : "Copy the highlighted text in the clipboard",             "paste" : "Paste the clipboard text to cursor location",             ...} 

What I would like to know, is if there is a method in sphinx to load the yaml file during the make html cycle, translate the python dictionary in some reStructuredText format (e.g. a definition list) and include in my html output.

I would expect my .rst file to look like:

Available commands ================== The commands available in bla-bla-bla...  .. magic-directive-that-execute-python-code::    :maybe python code or name of python file here: 

and to be converted internally to:

Available commands ================== The commands available in bla-bla-bla...  copy   Copy the highlighted text in the clipboard  paste   Paste the clipboard text to cursor location 

before being translated to HTML.

Answer 1

At the end I find a way to achieve what I wanted. Here's the how-to:

1. Create a python script (let's call it generate-includes.py) that will generate the reStructuredText and save it in the myrst.inc file. (In my example, this would be the script loading and parsing the YAML, but this is irrelevant). Make sure this file is executable!!!

2. Use the include directive in your main .rst document of your documentation, in the point where you want your dynamically-generated documentation to be inserted:

   .. include:: myrst.inc 

3. Modify the sphinx Makefile in order to generate the required .inc files at build time:

   myrst.inc:     ./generate-includes.py  html: myrst.inc     ...(other stuff here) 

4. Build your documentation normally with make html.

Answer 2

An improvement based on Michael's code and the built-in include directive:

import sys from os.path import basename  try:     from StringIO import StringIO except ImportError:     from io import StringIO  from docutils.parsers.rst import Directive     from docutils import nodes, statemachine  class ExecDirective(Directive):     """Execute the specified python code and insert the output into the document"""     has_content = True      def run(self):         oldStdout, sys.stdout = sys.stdout, StringIO()          tab_width = self.options.get('tab-width', self.state.document.settings.tab_width)         source = self.state_machine.input_lines.source(self.lineno - self.state_machine.input_offset - 1)          try:             exec('\n'.join(self.content))             text = sys.stdout.getvalue()             lines = statemachine.string2lines(text, tab_width, convert_whitespace=True)             self.state_machine.insert_input(lines, source)             return []         except Exception:             return [nodes.error(None, nodes.paragraph(text = "Unable to execute python code at %s:%d:" % (basename(source), self.lineno)), nodes.paragraph(text = str(sys.exc_info()[1])))]         finally:             sys.stdout = oldStdout  def setup(app):     app.add_directive('exec', ExecDirective) 

This one imports the output earlier so that it goes straight through the parser. It also works in Python 3.