Is there a way to log number of retries made

Question

I am trying to hit web-service which has Thread limit, i.e. if too many requests are made within seconds it will throw exception assuming it's an attack.

To resolve same we are using retrying module from python

Here is sample code

from retrying import retry


@retry(stop_max_attempt_number=10,wait_exponential_multiplier=1000, wait_exponential_max=10000)
def f():
    # Call web service 

This works however there is no messages given which indicate it's working.

Is there a parameter / way by which we could use to log number of retries made, something like waiting for 10 sec before next attempt

Answer 1

You can use the wait_func parameter for retry with a function that takes the number of attempts and the number of milliseconds of delay as parameters, and prints a message as desired:

from retrying import retry
import time

def wait(attempts, delay):
    print('Attempt #%d, retrying in %d seconds' % (attempts, delay // 1000))
    return delay

@retry(wait_func=wait)
def f():
    time.sleep(1)
    raise RuntimeError

f()

This outputs:

Attempt #1, retrying in 1 seconds
Attempt #2, retrying in 3 seconds
Attempt #3, retrying in 7 seconds
Attempt #4, retrying in 15 seconds
...

If you would like to use the exponential sleep option, however, you would have to instantiate your own Retrying instance and use a custom decorator to make use of it, so that your custom wait function can call the exponential_sleep method with your Retrying instance:

from retrying import Retrying

def wait(attempts, delay):
    print('Attempt #%d, retrying in %d seconds' % (attempts, delay // 1000))
    return retrying.exponential_sleep(attempts, delay)

retrying = Retrying(wait_func=wait, stop_max_attempt_number=10, wait_exponential_multiplier=1000, wait_exponential_max=10000)
custom_retry = lambda f: lambda *args, **kwargs: retrying.call(f, *args, **kwargs)

@custom_retry
def f():
    raise RuntimeError

f()

This outputs:

Attempt #1, retrying in 0 seconds
Attempt #2, retrying in 2 seconds
Attempt #3, retrying in 6 seconds
Attempt #4, retrying in 14 seconds
Attempt #5, retrying in 24 seconds
Attempt #6, retrying in 34 seconds
...

Answer 2

Here is another approach to resolve it

after_attempts as another flag which could be used

def _after(attempt_number):
    print(attempt_number)


@retry(wait_fixed=100, stop_max_attempt_number=3, after_attempts=_after)
def _test_after():
    raise Exception("testing after_attempts handler")

But this is still under development and could be fetched using retrying master code not part of latest release : 1.3.3