Sphinx documentation processor extension works differently for HTML and LaTeX output?

Question

I have a simple Sphinx extension as follows:

from docutils import nodes
from docutils.parsers.rst import directives
from sphinx.util.compat import Directive

class testnode(nodes.Element):
    def __init__(self, *args, **kwargs):
        super(testnode, self).__init__(*args, **kwargs)
        self['foo'] = '?'

def visit_testnode_latex(self, node):
    self.body.append('Test: %s' % node['foo'])

def depart_testnode_latex(self, node):
    pass

def visit_testnode_html(self, node):
    self.body.append('<p>Test: %s</p>' % node['foo'])

def depart_testnode_html(self, node):
    pass

class TestDirective(Directive):
    has_content = False
    required_arguments = 0
    optional_arguments = 0
    final_argument_whitespace = False
    option_spec = {
        'foo': directives.unchanged,
    }

    def run(self):
        node = testnode()
        node['foo'] = self.options.get('foo')
        return [node]

def setup(app):
    app.add_directive("testdirective", TestDirective)
    app.add_node(testnode,
                 html=(visit_testnode_html,
                       depart_testnode_html),
                 latex=(visit_testnode_latex,
                        depart_testnode_latex))

Given a document containing

.. testdirective::
   :foo: bar

the HTML output contains »Test: bar« but the LaTeX output contains »Test: ?« (the default value). I checked that node['foo'] has the correct value after the assignment in TestDirective.run(), but that does not appear to stick around until the LaTeX writer runs.

What am I doing wrong?

Answer 1

After stepping through the LaTeX writer for Sphinx I found the problem here. It's the way in which you're setting a default for the 'foo' keyword in the testnode initializer.

There's a point in the LaTeX writer that it does a deepcopy of your entire document tree in order to inline it in another tree. A deepcopy on an Element node initializes a new node of the same class and passes all the attributes and contents of the original node through the constructor. So when your testnode gets copied your constructor overrides the original 'foo' attribute that's passed into the constructor. Instead write it like this and it should work:

class testnode(nodes.Element):
    def __init__(self, *args, **kwargs):
        super(testnode, self).__init__(*args, **kwargs)
        if 'foo' not in self:
            self['foo'] = '?'

This will prevent your default from overriding any explicit value for the attribute that's been passed to the constructor. There are several other variations possible.