Inclusion of external documentation into a Sphinx project

Question

We maintain a fairly large documentation using Sphinx in SVN.

As part of the generated output we would like to include the release notes of related Python modules as primary content (not as hyperlink!). The release notes of the external modules are also maintained in SVN. Is there some Sphinx-ish way to pull in the parts of the documentation from other (SVN) sources? Ok, using SVN externals is a way to solve the problem but perhaps not the smartest way...any better options?

Answer 1

The two options I can think of are:

1. Add an svn:externals link to the remote project (which you already know about).
2. Extend Sphinx with a custom directive to include files from remote subversion repositories.

I'm no expert on Sphinx internals but was able to cobble together a quick extension which embeds files from a remote subversion repository.

The extension adds an svninclude directive which takes 1 argument, the url of the repository where your docs are located. It checks this repository out into a temp directory _svncache located in the project root, and then proceeds to read the contents of each file and insert them into the parser's state machine.

Here is the code for the svninclude.py extension. It is oversimplified and has no error checking at the moment. If you plan to implement this let me know and I can provide some additional tips if you get stuck:

import os, re, subprocess, sys
from docutils import nodes, statemachine
from docutils.parsers.rst import directives
from sphinx.util.compat import Directive, directive_dwim

class SvnInclude(Directive):

    has_content = True
    required_arguments = 1
    optional_arguments = 0
    final_argument_whitespace = False

    def _setup_repo(self, repo):
        env = self.state.document.settings.env
        path = os.path.normpath(env.doc2path(env.docname, base=None))
        cache = os.path.join(os.path.dirname(path), '_svncache')
        root = os.path.join(cache, re.sub('[\W\-]+', '_', repo))
        if not os.path.exists(root):
            os.makedirs(root)
        subprocess.call(['svn', 'co', repo, root])
        return root

    def run(self):
        root = self._setup_repo(self.arguments[0])
        for path in self.content:
            data = open(os.path.join(root, path), 'rb').read()
            lines = statemachine.string2lines(data)
            self.state_machine.insert_input(lines, path)
        return []

def setup(app):
    app.add_directive('svninclude', directive_dwim(SvnInclude))

Here is an example of the markup you'd include in your index.rst (or other file):

.. svninclude:: http://svn.domain.com/svn/project

    one.rst
    doc/two.rst

Where the paths one.rst and doc/two.rst are relative to the subversion url, for example http://svn.domain.com/svn/project/one.rst.

You'd of course want to package up the svninclude.py and get it installed in your Python path. Here's what I did to test it:

1. Added 'svninclude' to the extensions list in source/conf.py.
2. Placed svninclude.py in the project root.

Then ran:

% PYTHONPATH=. sphinx-build -b html ./source ./build