Modifying global variables in doctests

Question

The doctest documentation has a section about execution context. My reading is that the globals in the module are shallow copied for the tests in each docstring, but are not reset between tests within a docstring.

Based on that description, I would have thought the following doctests would pass:

X = 1


def f():
    """Function F.

    >>> X
    1
    >>> f()
    2
    >>> X
    2
    """
    global X
    X = 2
    return X


def g():
    """Function G.

    >>> g()
    1
    >>> X
    1
    """
    return X

But instead the following tests pass!

X = 1


def f():
    """Function F.

    >>> X
    1
    >>> f()
    2
    >>> X
    1
    """
    global X
    X = 2
    return X


def g():
    """Function G.

    >>> g()
    2
    >>> X
    1
    """
    return X

It seems like the globals are shared across tests across docstrings? But only within function calls?

Why is this the resulting behavior? Does this have something to do with functions having a dictionary of globals separate from the execution context?

Answer 1

Not exactly. While it is true that the globals are shallow copied, what you are actually seeing is the scoping of globals (using the keyword global) and how it actually operates at the module level in Python. You can observe this by putting in a pdb.set_trace() inside function f right after the assignment (X = 2).

$ python -m doctest foo.py
> /tmp/foo.py(18)f()
-> return X
(Pdb) bt
  /usr/lib/python2.7/runpy.py(162)_run_module_as_main()
-> "__main__", fname, loader, pkg_name)
  /usr/lib/python2.7/runpy.py(72)_run_code()
-> exec code in run_globals
  /usr/lib/python2.7/doctest.py(2817)<module>()
-> sys.exit(_test())
  /usr/lib/python2.7/doctest.py(2808)_test()
-> failures, _ = testmod(m)
  /usr/lib/python2.7/doctest.py(1911)testmod()
-> runner.run(test)
  /usr/lib/python2.7/doctest.py(1454)run()
-> return self.__run(test, compileflags, out)
  /usr/lib/python2.7/doctest.py(1315)__run()
-> compileflags, 1) in test.globs
  <doctest foo.f[1]>(1)<module>()
-> f()
> /tmp/foo.py(18)f()
-> return X
(Pdb) pp X
2

Yes, the value is indeed 2 within the scope of f, but let's take a look of its globals. Let's look at how they compare in the current frame and up a frame.

(Pdb) id(globals())
140653053803048  # remember this number, and we go up a frame
(Pdb) u
> <doctest foo.f[1]>(1)<module>()
-> f()
(Pdb) id(globals())
140653053878632  # the "shallow" clone
(Pdb) X
1
(Pdb) c

Ah ha, you can see that they are NOT actually the same thing, and that X is indeed 1 and has not been changed, because the globals there are within the <doctest doc.f> module created by doctest for this reason. Let's continue.

(Pdb) id(globals())
140653053803048  # hey look, is the SAME number we remember
(Pdb) u
> <doctest foo.g[0]>(1)<module>()
-> g()
(Pdb) id(globals())
140653053872960  # note how this is a different shallow clone

So what you actually saw is that the globals within the doctest is not the same one as the one on your source (hence g will return 2 because X was really was changed in the module here by f, but not in the doctest module shallow copy scope), even though it originally was copied from the module but the changes are not reflected back to the underlying module since this is how the global keyword operates - at the module level, not across modules.