How to unit-test decorated functions?

Question

I tried lately to train myself a lot in unit-testing best practices. Most of it makes perfect sense, but there is something that is often overlooked and/or badly explained: how should one unit-test decorated functions ?

Let's assume I have this code:

def stringify(func):
    @wraps(func)
    def wrapper(*args):
        return str(func(*args))

    return wrapper


class A(object):
    @stringify
    def add_numbers(self, a, b):
        """
        Returns the sum of `a` and `b` as a string.
        """
        return a + b

I can obviously write the following tests:

def test_stringify():
    @stringify
    def func(x):
        return x

    assert func(42) == "42"

def test_A_add_numbers():
    instance = MagicMock(spec=A)
    result = A.add_numbers.__wrapped__(instance, 3, 7)
    assert result == 10

This gives me 100% coverage: I know that any function that gets decorated with stringify() gets his result as a string, and I know that the undecorated A.add_numbers() function returns the sum of its arguments. So by transitivity, the decorated version of A.add_numbers() must return the sum of its argument, as a string. All seems good !

However I'm not entirely satisfied with this: my tests, as I wrote them could still pass if I were to use another decorator (that does something else, say multiply the result by 2 instead of casting to a str). My function A.add_numbers would not be correct anymore yet the tests would still pass. Not awesome.

I could test the decorated version of A.add_numbers() but then I would overtest things since my decorator is already unit-tested.

It feels like I'm missing something here. What is a good strategy to unit-test decorated functions ?

Answer 1

I ended up splitting my decorators in two. So instead of having:

def stringify(func):
    @wraps(func)
    def wrapper(*args):
        return str(func(*args))

    return wrapper

I have:

def to_string(value):
    return str(value)

def stringify(func):
    @wraps(func)
    def wrapper(*args):
        return to_string(func(*args))

    return wrapper

Which allows me later to simply mock-out to_string when testing the decorated function.

Obviously in this simple example case it might seem overkill, but when used over a decorator that actually does something complex or expensive (like opening a connection to a DB, or whatever), being able to mock it out is a very nice thing.

Answer 2

Test the public interface of your code. If you only expect people to call the decorated functions, then that's what you should test. If the decorator is also public, then test that too (like you did with test_stringify()). Don't test the wrapped versions unless people are directly calling them.

Answer 3

One of the major benefits of unit testing is to allow refactoring with some degree of confidence that the refactored code continues to work the same as it did previously. Suppose you had started with

def add_numbers(a, b):
    return str(a + b)

def mult_numbers(a, b):
    return str(a * b)

You would have some tests like

def test_add_numbers():
    assert add_numbers(3, 5) == "8"

def test_mult_numbers():
    assert mult_numbers(3, 5) == "15"

Now, you decide to refactor the common parts of each function (wrapping the output in a string), using your stringify decorator.

def stringify(func):
    @wraps(func)
    def wrapper(*args):
        return str(func(*args))

    return wrapper

@stringify
def add_numbers(a, b):
    return a + b

@stringify
def mult_numbers(a, b):
    return a * b

You'll notice that your original tests continue to work after this refactoring. It doesn't matter how you implemented add_numbers and mult_numbers; what matters is they continue to work as defined: returing a stringified result of the desired operation.

The only remaining test you need to write is one to verify that stringify does what it is intended to do: return the result of the decorated function as a string, which your test_stringify does.

---

Your issue seems to be that you want to treat the unwrapped function, the decorator, and the wrapped function as units. But if that's the case, then you are missing one unit test: the one that actually runs add_wrapper and tests its output, rather than just add_wrapper.__wrapped__. It doesn't really matter if you consider testing the wrapped function as a unit test or an integration test, but whatever you call it, you need to write it, because as you pointed out, it's not sufficient to test just the unwrapped function and the decorator separately.