Python: How to try again when experiencing a timeout

Question

I have a Python program that connects to a server to send it some commands, but occasionally I get this error:

TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

In previous situations like this, I would use something like this:

try:
            Do something
        except KeyError:
            do something else

Could I do the same thing in this same situation? I.e.,

try:
                Do something
            except TimeoutError:
               Do something again

And if so, what would I do after the except TimeoutError? Would I just do the same command again?

Answer 1

Could I do the same thing in this same situtation

Yes! You can use try/except for any exception, and TimeoutError is nothing special.

and if so what would I do after the except TimeoutError? would I just do the same command again?

If you only want to retry once, and let a second timeout count as a real error, yes:

try:
    do something
except TimeoutError:
    do something

(If "do something" is more than a simple statement, you probably want to factor out the code so you don't repeat yourself.)

However, if you want to retry multiple times, you probably want a loop:

for _ in range(max_retries):
    try:
        do something
        break
    except TimeoutError:
        pass

You may want to add an else clause to the for to distinguish the two cases (succeeded, and did a break, vs. never succeeded and just ran out of tries).

---

Since the idea here is usually to deal with possibly-transient errors, there are additional things you might want to add, such as:

• Re-raising the error, or raising a different one, after max_retries.
• Logging at progressively higher levels (e.g., a debug message for a single failure, but a warning for `max_retries).
• Retrying with exponential backoff (wait 1 second, then 2, 4, 8, …).
• Pushing the URL to the end of the work queue instead of retrying immediately. You can use (URL, retry_count) pairs if you also want max_retries logic, (URL, timestamp) pairs if you also want exponential backoff, or both if you want both. (Of course this only works if you don't care about the order of responses, or can reorder them at the end.)
• Different rules for different exceptions (e.g., 500, 502, 503, and 504 errors can all be caused by overload on a server or proxy, but the best retry logic may be different—and the best heuristics for 2018 may be different from 2010 or 2025).

For complicated combinations, a retry decorator, like the one linked in jterrace's helpful answer, is a great way to wrap up the behavior.

Answer 2

You can catch the TimeoutError like you mentioned:

import socket
import sys
try:
  dosomething()
except socket.TimeoutError:
  print >> sys.stderr, 'Retrying after TimeoutError'
  dosomething()

You could also use the retry decorator pattern on a function:

@retry(socket.TimeoutError)
def dosomething():
    # code that causes a TimeoutError
    ...