Multiple exceptions and code coverage when unit testing python

Question

The Problem:

Here is an artificial example of the code under test:

from datetime import datetime

def f(s):
    try:
        date = s.split(":")[1]
        return datetime.strptime(date, "%Y%m%d")
    except (ValueError, IndexError) as e:
        # some code here
        raise

Here is a set of tests I currently have:

from datetime import datetime
import unittest

from test_module import f

class MyTestCase(unittest.TestCase):
    def test_valid_date(self):
        self.assertEqual(f("1:20130101"), datetime(2013, 1, 1))

    def test_invalid_date(self):
        self.assertRaises(ValueError, f, "1:invalid")

The test passes and, if I run the coverage with the --branch flag, I would get 100% line and branch coverage:

$ coverage run --branch -m unittest test
..
----------------------------------------------------------------------
Ran 2 tests in 0.003s

OK
$ coverage report
Name            Stmts   Miss Branch BrPart  Cover
--------------------------------------------
test_module.py      7      0      0      0   100%
--------------------------------------------
TOTAL               7      0      0      0   100%

However, note that the test currently examines only two cases - when there is no exception thrown, and there is a ValueError exception raised.

The question:

Is there a way for coverage to report that I have not tested a case when IndexError is raised?

Answer 1

Coverage.py can only measure which execution paths (statements or branches) were run. It has no means of tracking what values were used, including what exception types were raised.

As I see it, your options are:

1. Separate the exception clauses. In the code you've shown, the two exceptions could be raised by separate lines anyway, though perhaps in your real code they are not so separable.

2. Don't worry about the two exceptions. Your tests for this code will likely consider a number of different inputs, designed to exercise different edge cases. Coverage.py can't help you with distinguishing among them all, or ensuring that you've written enough cases. Use other criteria to decide you've written enough test cases.