External library mock patch used in all tests doesnt work when tests run together

Question

I am using Python's mock library along with unittest. I am writing unit tests for a class that uses a function of an external library in one of its methods. Depending on the case, this function returns different values.

So let's say I wanna test class A:

from external_library import function_foo

class A(object):
...

In my test class, in order to use the values returned by the function from the external library, I create a patch, and only import class A after defining the patch. However, I need to use this function in all my test methods, and in each method it returns different values.

My test class is as follows:

class TestA(TestCase):

    @patch('external_library.function_foo', side_effect=[1, 2, 3])    
    def test_1(self, *patches):

       from module import class A
       obj = A()
       ...

    @patch('external_library.function_foo', side_effect=[1, 1, 2, 2, 3, 3])    
    def test_2(self, *patches):

       from module import class A
       obj = A()
       ...

    ...

I have 10 tests and only 1 (the first one) passes when I run all of them together, for the rest, I get StopIteration error. However, if I run each one of them individually, they all pass.

I have tried using with patch('external_library.function_foo', side_effect=[...]) in each method, but the outcome was the same. I also tried creating only once the patch in the setUp method, starting it, reassigning the side_effect within each method, and stopping in tearDown, but it didn't work.

Any ideas on what might work in this case?

Thanks!

Answer 1

The caveat is, the second time you import a module, it would not be loaded again, you get the same module object as the first time you imported.

When you first run "test_1", external_library.function_foo replaced by a Mock object, let's name it mock_a. Then your "module" get imported for the first time, python will load it, means, execute code inside "module", which will bind name "function_foo" to object "mock_a" in the namespace of "module", save "module" object to sys.modules. This time your test will pass, and side_effect of mock_a get consumed.

Next is "test_2", external_library.function_foo replaced by a Mock object, name it to mock_b. Then import "module", this time it would not be loaded again, but populate from sys.modules, you get the same module object as in "test_1". In the namespace of this module object, name "function_foo" is still bound to object mock_a, not the newly created mock_b. Because side_effect of mock_a has already consumed, StopIteration error raised.

You should apply patch to where a name is looked up, but not where it is defined:

@patch('module.function_foo', side_effect=[1, 2, 3])    
def test_1(self, patch):
    ...

Read more detail on the "Where to patch" section of the manual of patch.