How to remove the effects of a decorator while testing in python? [duplicate]

Question

I'm using the retry decorator in some code in python. But I want to speed up my tests by removing its effect.

My code is:

@retry(subprocess.CalledProcessError, tries=5, delay=1, backoff=2, logger=logger)
def _sftp_command_with_retries(command, pem_path, user_at_host):
    # connect to sftp, blah blah blah
    pass

How can I remove the effect of the decorator while testing? I can't create an undecorated version because I'm testing higher-level functions that use this.

Since retry uses time.sleep to back off, ideally I'd be able to patch time.sleep but since this is in a decorator I don't think that's possible.

Is there any way I can speed up testing code that uses this function?

Update

I'm basically trying to test my higher-level functions that use this to make sure that they catch any exceptions thrown by _sftp_command_with_retries. Since the retry decorator will propagate them I need a more complicated mock.

So from here I can see how to mock a decorator. But now I need to know how to write a mock that is itself a decorator. It needs to call _sftp_command_with_retries and if it raises an exception, propagate it, otherwise return the return value.

Adding this after importing my function didn't work:

_sftp_command_with_retries = _sftp_command_with_retries.__wrapped__ 

Answer 1

The retry decorator you are using is built on top of the decorator.decorator utility decorator with a simpler fallback if that package is not installed.

The result has a __wrapped__ attribute that gives you access to the original function:

orig = _sftp_command_with_retries.__wrapped__

If decorator is not installed and you are using a Python version before 3.2, that attribute won't be present; you'd have to manually reach into the decorator closure:

orig = _sftp_command_with_retries.__closure__[1].cell_contents

(the closure at index 0 is the retry_decorator produced when calling retry() itself).

Note that decorator is listed as a dependency in the retry package metadata, and if you installed it with pip the decorator package would have been installed automatically.

You can support both possibilities with a try...except:

try:
    orig = _sftp_command_with_retries.__wrapped__
except AttributeError:
    # decorator.decorator not available and not Python 3.2 or newer.
    orig = _sftp_command_with_retries.__closure__[1].cell_contents

Note that you always can patch time.sleep() with a mock. The decorator code will use the mock as it references the 'global' time module in the module source code.

Alternatively, you could patch retry.api.__retry_internal with:

import retry.api
def dontretry(f, *args, **kw):
    return f()

with mock.patch.object(retry.api, '__retry_internal', dontretry):
    # use your decorated method

This temporarily replaces the function that does the actual retrying with one that just calls your original function directly.