Testing warnings with doctest

Question

I'd like to use doctests to test the presence of certain warnings. For example, suppose I have the following module:

from warnings import warn

class Foo(object):
    """
    Instantiating Foo always gives a warning:

    >>> foo = Foo()
    testdocs.py:14: UserWarning: Boo!
      warn("Boo!", UserWarning)
    >>> 
    """

    def __init__(self):
        warn("Boo!", UserWarning)

If I run python -m doctest testdocs.py to run the doctest in my class and make sure that the warning is printed, I get:

testdocs.py:14: UserWarning: Boo!
  warn("Boo!", UserWarning)
**********************************************************************
File "testdocs.py", line 7, in testdocs.Foo
Failed example:
    foo = Foo()
Expected:
    testdocs.py:14: UserWarning: Boo!
      warn("Boo!", UserWarning)
Got nothing
**********************************************************************
1 items had failures:
   1 of   1 in testdocs.Foo
***Test Failed*** 1 failures.

It looks like the warning is getting printed but not captured or noticed by doctest. I'm guessing that this is because warnings are printed to sys.stderr instead of sys.stdout. But this happens even when I say sys.stderr = sys.stdout at the end of my module.

So is there any way to use doctests to test for warnings? I can find no mention of this one way or the other in the documentation or in my Google searching.

Answer 1

This isn't the most elegant way to do it, but it works for me:

from warnings import warn

class Foo(object):
    """
    Instantiating Foo always gives a warning:

    >>> import sys; sys.stderr = sys.stdout
    >>> foo = Foo() # doctest:+ELLIPSIS
    /.../testdocs.py:14: UserWarning: Boo!
      warn("Boo!", UserWarning)
    """

    def __init__(self):
        warn("Boo!", UserWarning)

if __name__ == '__main__':
    import doctest
    doctest.testmod()

This presumably won't work on Windows, though, since the path reported in the UserWarning output must start with a slash the way I've written this test. You may be able to figure out some better incantation of the ELLIPSIS directive, but I could not.

Answer 2

The Testing Warnings sections of the Python documentation is dedicated to this topic. However, to summarize, you have two options:

(A) Use the catch_warnings context manager

This is recommended course in the official documentation. However, the catch_warnings context manager only came into existence with Python 2.6.

import warnings

def fxn():
    warnings.warn("deprecated", DeprecationWarning)

with warnings.catch_warnings(record=True) as w:
    # Cause all warnings to always be triggered.
    warnings.simplefilter("always")
    # Trigger a warning.
    fxn()
    # Verify some things
    assert len(w) == 1
    assert issubclass(w[-1].category, DeprecationWarning)
    assert "deprecated" in str(w[-1].message)

(B) Upgrade Warnings to Errors

If the warning hasn't been seen before— and thus was registered in the warnings registry— then you can set warnings to raise exceptions and catch it.

import warnings


def fxn():
    warnings.warn("deprecated", DeprecationWarning)


if __name__ == '__main__':
    warnings.simplefilter("error", DeprecationWarning)

    try:
        fxn()
    except DeprecationWarning:
        print "Pass"
    else:
        print "Fail"
    finally:
        warnings.simplefilter("default", DeprecationWarning)