passing generic exception to retrying module

Question

I am hitting an external api that allows 1request/sec so to avoid request collision, I am using retrying module in python. I want to retry on generic exception and try with a random time frame.

@retry(retry_on_exception=exceptions.Exception,wrap_exception=True,wait_random_min=1000,wait_random_max=2500)
def do_some _stuff(info):
      #some stuff that can throw exception

Does it seem like a robust solution to my problem, I am not sure? Also, is it the correct way to handle generic exception in retry decorator. I am using it exactly like the above code and it does not throw any errors but not sure. All the examples I have seen are with some specific exceptions.

updated: Exceptions I am getting

reject |= self._retry_on_exception(attempt.value[1])
TypeError: unsupported operand type(s) for |=: 'bool' and 'exceptions.Exception'

Answer 1

You're not using the retry_on_exception parameter correctly. It expects a callable and that callable must return a boolean. You can see an example of this in the package doc:

def retry_if_io_error(exception):
    """Return True if we should retry (in this case when it's an IOError), False otherwise"""
    return isinstance(exception, IOError)

@retry(retry_on_exception=retry_if_io_error)
def might_io_error():
    print "Retry forever with no wait if an IOError occurs, raise any other errors"

So in your case you can check on Exception instances, instead of IOError, with the test isinstance(exception, Exception). But you can notice that it'll always be true and so it's sort of pointless. @retry default behavior is to retry independently from what the exception is, so you actually have nothing to add:

@retry(wait_random_min=1000, wait_random_max=2500)
def do_some_stuff(info):
      #some stuff that can throw exception

But it's generally a bad idea to treat all types of exceptions the same way. In your case it looks like you should only retry on some HttpError or whatever similar error your code raises when you hit the API limit.