Is the interaction between python unittest subTest and skipTest defined?

Question

I've not found a definitive answer regarding the interaction between Python unittest features subTest and skipTest.

For instance, testing it, it seems that when calling skipTest within a subTest context (new in Python 3.4), only the current subTest is skipped. Allowing any loop around the subTest to continue with other values.

from unittest import TestCase

class NumbersTest(TestCase):
    def test_even(self):
        """
        Test that numbers between 0 and 5 are all even.
        """
        for i in range(0, 6):
            with self.subTest(i=i):
                if i==3:
                    self.skipTest("Skip 3.")
                self.assertEqual( i%2, 0 )

Output shows that test continues with values 4 and 5 after skipping 3.

test_even (TestUnitSubTest.NumbersTest) ... skipped 'Skip 3.'

======================================================================
FAIL: test_even (TestUnitSubTest.NumbersTest) (i=1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "~/TestUnitSubTest.py", line 14, in test_even
    self.assertEqual( i%2, 0 )
AssertionError: 1 != 0

======================================================================
FAIL: test_even (TestUnitSubTest.NumbersTest) (i=5)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "~/TestUnitSubTest.py", line 14, in test_even
    self.assertEqual( i%2, 0 )
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 1 test in 0.002s

FAILED (failures=2, skipped=1)

I've not found the definitive answer in Python documentation. It states, by example, that if an assertion fails within a subTest context, then further subTest will execute. But I didn't find anything regarding skipping a test.

Can anyone point me to the paragraph I missed?

Answer 1

I don't have an answer to this per-se, but four years later, ran into the same issue as this question.

It looks like .subTest() and .skipTest() interaction (as well as general logging of subtests) hasn't been fully worked out. https://bugs.python.org/issue25894 and https://bugs.python.org/issue35327 are two open bugs relating to this exact question, opened since 2015 or so.

As of 2020, it looks like both are still open, so this made not be a solved thing yet.