How should I test using Mocks in Python?

Question

I can see two different approaches to injecting mocks into python code that I want to test:

1. Dependency Injection:

   Allow the collaborating classes to passed into the constructor of the object under test, and pass in mock objects (and factories where necessary, like Java)

2. Monkey Patching:

   Stub out the collaborating classes in the module under test with a mock object factory (so that constructing the collaborating classes actually creates a mock object). I don't need to allow them to be injected through the constructor or create any factories.

Both of these approaches seem to be supported by the python mocking libraries (for example, mox or mock). Which approach should I use in Python, is one of these sensible or is there a better approach?

Answer 1

Note: Any answer to this question is going to be equally applicable to passing other kinds of doubles (mocks, fakes, stubs, etc.).

There's a lot of religion regarding this topic, so the standard answer to this question is: 'Do what's practical for your application and use your sense of code smell.' While I'm also inclined to reject the either-or approach, this answer strikes me as inherently useless to anyone actually asking the question (namely myself). Here's the decision process I'm using, and some of the considerations I made in it's development:

Prior Evaluation

1. Can your code benefit from more functional programming? If you can factor out dependencies and side effects, you don't need to mock anything.
2. Can the unit be meaningfully unit tested? Maybe it needs to be refactored, or maybe it's just "glue code" that can't be fruitfully unit tested (but must be covered by system tests).
3. Do you really need to mock the dependency? If running your real code would still allow you to meaningfully test the behavior and would be neither slow nor destructive, let it run.

Mocking Strategy

• If the particular dependency is essential to the unit, hardcode it in the product and monkey patch in test.
  • Exception: If using the production dependency in test could be harmful to your system, inject it as a positional argument.
  • Exception: If there’s a risk of the monkey patch applying in another of the unit’s dependencies and that would be undesirable, inject it as a positional argument.
• If a dependency of the same kind is essential to the unit, inject it as a required, positional argument.
• If the dependency is non-essential to the unit, inject it as an optional, keyword argument.

---

Terminology

Dependency Injection: In the context of Python, this term is usually refers specifically to constructor injection.

Monkey Patching: Binding a name (in the code under test) to a different object in runtime than it is bound to in the module. In practice, this often means using mock.patch.

Example

Say we've got a function with a side effect that is undesirable during testing, be it destructive (writing nonsense to our production database) or annoying (i.e., slow). Here's an example of the latter case and its test:

def foo():
    ...
    time.sleep(5)
    ...
    return "bar"

...

def foo_test():
    assertEqual(foo(), "bar")

Our test works, but takes a minimum of five seconds. We can avoid the wait by replacing time.sleep with a mock object that does nothing. The two strategies of doing this are the topic of this question:

Dependency Injection

def foo(wait=time.sleep):
    ...
    wait(5)
    ...
    return "bar"

...

def foo_test():
    assertEqual(foo(wait=mock.Mock()), "bar")

Monkey Patching

def foo():
    ...
    time.sleep(5)
    ...
    return "bar"

...

@mock.patch('time.sleep')
def foo_test():
    assertEqual(foo(), "bar")

Why Dependency Injection?

I find the pros and cons of monkey patching more straightforward, so I focused my analysis on dependency injection.

Cluttering Interfaces Just for Testability?

Dependency injection is very explicit, but requires modification of the product code. Monkey patching is not explicit, but does not require modification of the product code.

The programmer's gut reaction is to make many sacrifices before modifying the product code for tests. After considering what your application would look like with all dependencies injected, preference for monkey patching seems like a no-brainer. As Michael Foord expresses:

[E]ven internal APIs still need to be read / used by developers and I don't like screwing with them either.

...

My contention is that dependency injection just for testability is never required for Python and rarely preferable to other techniques. There are plenty of times when dependency injection is useful as a structure / architecture in its own right though.

While the topic naturally comes up when writing unit tests, a charitable interpretation of those who have advocated dependency injection concludes that testability is not their primary motivation. Ned Batchelder finds (25:30) that monkey patching "makes the code kind of difficult to understand, and I'd rather somewhere see: we're testing now, so this is how you get time." He elaborates (3:14):

When I sit down and look at my code and think, 'How could I test this better?' and I change the product code to make it more testable, it's actually better product code. And I think that's because, if you have to write something that does one thing, it can do that one thing well, but if you write something that can do two things well, that's a better thing. And doing testing well, in addition to being a good product, makes the code better. By having two uses, you have to really think about that API, and you have to really think about what that one piece of code is doing. And by having it do both of those things well, you've got a better, modular, more abstraction design, that's going to be better for your product in the long run. So most of what you do for testability is going to be good for the product in addition to all the other good things about testability--finding more bugs and having better quality and all that stuff.

The Interface Pollution Problem

No, not just visual pollution. Let's say we realize over time that in a corner case we need a more complex algorithm to determine the wait time in our foo function above:

-- bar = foo()
++ bar = foo(wait=mywait)

But some time later the waiting becomes unnecessary for our primary uses of foo. We use the dependency injection pattern all the time, so we assume we can remove named arguments without consequence.

-- def foo(wait=time.sleep):
++ def foo():

We now need to track down our corner cases to avoid a TypeError. It seems like even though these arguments were only added for testing purposes they are going to be used in production, and that this approach limits your ability to refactor by placing an implementation detail in the interface.

But Aaron Maxwell has a solution:

In real code I tend to mark "test hook only" parameters with an underscore prefix - so the signature would be: __init__(self, rel_url, _urlopen=urlopen) And then in the docstring for the method, I make clear it's a testing hook that is liable to go away without warning. (Yes, I'll be sure to write a docstring in this case :) The underscore is just my way of highlighting the parameter as special in some way.

Of course, that's if I want it to only be used for testing. If it is something we decide we want to make available outside of that context, and commit to keeping around, I won't put up "keep off" signs like that :)

While this approach does address the pollution concern, to me, all this clutter--first adding to the interface and second ensuring that you don't actually use the interface--has a smell.

But Augie Fackler and Nathaniel Manista's position that required, positional arguments are safer than optional, keyword arguments, would make the pollution problem moot. They elaborate:

If it's controlling a critical piece of behavior, like where it's going to write its persistent data, we've found that it's much safer to make it a required argument and just always specify it. We've found that in cases of object relationship where the first object does not make sense unless it also has a second--so, a user profile doesn't make sense unless it has a user credential--we've found that explicit construction parameters is the most robust solution for us...[Optional parameters] are good for things that change the behavior of your object in a subtle way.

Without getting caught up in assessing their broader testing strategy, the point we should easily assent to is that critical components should not be passed as optional parameters.

However, I don't see why "critical" dependencies shouldn't be hard-coded when the relationship is with a particular dependency. An abstraction's substance is it's relationship to other abstractions. So if an essential property of an abstraction is a relationship with another abstraction within your application, it's a prime candidate for hard-coding--no matter how much the implementation details change within either abstraction, they are permanently coupled.

Part of the distinction being made is between dependencies that pose a risk to the system and those that do not. If the dependency is responsible for writing to a database, pushing to clients, or dropping a bomb, then unless we're writing drone software we can't afford to make mistakes.

It's worth noting that injecting with positional arguments makes the "wait-and-see" strategy costly. If we decide that one of our hard-coded dependencies needs to be selected in the constructor some day, adding it as a positional parameter would break backwards compatibility. A similar problem would be encountered if we later decided to remove a required parameter, so non-essential dependencies must be optional parameters so we have the freedom to change the interface.

Dependencies are bad, mkay?

Constructor injection is one of several methods of dependency injection. According to Wikipedia: "Dependency injection is a software design pattern that implements inversion of control and allows a program design to follow the dependency inversion principle."

Inversion of Control serves the following design purposes:

• To decouple the execution of a task from implementation.
• To focus a module on the task it is designed for.
• To free modules from assumptions about how other systems do what they do and instead rely on contracts.
• To prevent side effects when replacing a module.

.

The goal of the dependency inversion principle is to decouple application glue code from application logic...

The principle states:

A. High-level modules should not depend on low-level modules. Both should depend on abstractions. B. Abstractions should not depend on details. Details should depend on abstractions.

The principle inverts the way some people may think about object-oriented design, dictating that both high- and low-level objects must depend on the same abstraction.

That's about as specific as I want to get with this terminology, given the disputed status of it's value. But it is these types of concerns that motivate Martelli (evidenced by his 2007 talk especially).

The advantages of dependency injection, can be distilled to reusability. Whether it be through global configuration, dynamic algorithms, or evolving application development, decoupling the abstraction of a function/method/class from the implementation details of it's dependencies allows each of these components (though in this case, particularly the abstraction) the potential to be coupled in combinations that were unplanned when it was written, without modification. Testing is a case in point, because testability is reusability.

Remember that gut reaction against changing product code to meet the needs of test code? Well you should have the same reaction to changing your abstractions to meet the particular needs of production implementations as well!

The practical takeaway from all this theory is to separate "glue code", which cannot be unit tested, from logic, which is what you want to unit test. While they're particularly interested in a specific implementation of this principle, I think Fackler and Manista give a good example of this. Where one might have had:

class OldClass(object):
    ...
    def EnLolCat(self, misspelt_phrases):
        lolcats = []
        for album in self.albums:
            for photo in album.Photos():
                exif = photo.EXIF()
                for text in misspelt_phrases:
                    geoText = text.Geo(exif)
                    if photo.canCat(geoText)
                        lolcat = photo.Cat(geoText)
                        self.lolcats = lolcats

They would suggest:

def Lol(albums, misspelt_phrases):
    lolcats = []
    for album in albums:
        for photo in album.Photos():
            exif = photo.EXIF()
            for text in misspelt_phrases:
                geoText = text.Geo(exif)
                if photo.canCat(geoText)
                    lolcat = photo.Cat(geoText)
                    return lolcats

class NewClass(object):
    ...
    def EnLolCat(self, misspelt_phrases):
        self.lolcats = Lol(
             self.albums, misspelt_phrases)

Where we might have found ourselves mocking the object instance in order to test EnLolCat, we now find ourselves with glue code in our class, and a free function we can easily test because it has no side effects and is completely deterministic. In other words, we're doing more functional programming.

But isn't our situation the same when it comes to testing the NewClass method? Not necessarily.

I believe in strong testing of behavioral parts of software, so things like functions, things like computations, things like state changes, and I don't believe in unit tests for assemblies: when you're starting up your application, when you're connecting one object to another, or when you're constructing something--what you might think of as glue code. It's simple enough, and it's going to be covered by your integration tests. This contemporary class example (NewClass): we might not write a test for that because there's really no interesting logic in there other than the side effect of setting an attribute...We've already written a test for the pure function that we factored out of that method, so there's not a whole lot of incremental benefit to testing this instance method as well.

Dependencies are bad for code under test, so maybe it's a good thing if each one makes your code a little uglier. The cost of ugliness is lower than the cost of tight coupling, yet something developers are more likely to take seriously.

Final Notes

• Even Alex Martelli monkey patches in emergencies.
• Even Michael Foord doesn't mock components (22:00) unless he really needs to.