How to handle a connection error gracefully in requests?

Question

I have a simple python telegram bot, here's the code:

import requests
import json
from time import sleep
import os

filename = 'bot_last_update'
target = open(filename, 'r')
update_from_file = target.read()

# check update from file
update_from_file = update_from_file.strip()
last_update = int(update_from_file)

token = xxxx
url = 'https://api.telegram.org/bot%s/' % token

# We want to keep checking for updates. So this must be a never ending loop
while True:
    # My chat is up and running, I need to maintain it! Get me all chat updates
    get_updates = json.loads(requests.get(url + 'getUpdates').content)
    # Ok, I've got 'em. Let's iterate through each one
    for update in get_updates['result']:
        # First make sure I haven't read this update yet
        if last_update < update['update_id']:
            last_update = update['update_id']
            target = open(filename, 'w')
            target.truncate()
            target.write(str(last_update))
            target.close()
            if update['message']['chat']['type'] == 'private':
            # I've got a new update. Let's see what it is.
                if update['message']['text'] == 'do something':
                    requests.get(url + 'sendMessage', params=dict(chat_id=update['message']['chat']['id'], text='doing it'))
                    os.system('/srv/scripts/do_something.sh')
                    sleep(10)
                    requests.get(url + 'sendMessage', params=dict(chat_id=update['message']['chat']['id'], text='done!'))
                else:
                    pass
    # Let's wait a few seconds for new updates
    sleep(1)

It works fine but every time I have some problem in my network I have this error:

Traceback (most recent call last):
  File "my_telegram_bot.py", line 21, in <module>
    get_updates = json.loads(requests.get(url + 'getUpdates').content)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 70, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(113, 'No route to host'))

What would be the best way to avoid that error ? I would like to keep this bot up at all times, so it should not fail in a critical way during those kinds of events (or if it does, it should auto-recover / restart by itself).

Answer 1

You need to implement a retry mechanism. Here is an example in python How to retry after exception in python?. A retry mechanism will keep the bot up and avoid the error assuming the connection corrects itself in a reasonable amount of time.

Check out Python requests exception handling for an example of catching your specific exception.

Combining the two examples we get:

from requests import ConnectionError
import requests
import json
import time
import os
connection_timeout = 30 # seconds

...

# My chat is up and running, I need to maintain it! Get me all chat updates
start_time = time.time()
while True:
    try:
        get_updates = json.loads(requests.get(url + 'getUpdates').content)
        break
    except ConnectionError:
        if time.time() > start_time + connection_timeout:
            raise Exception('Unable to get updates after {} seconds of ConnectionErrors'.format(connection_timeout))
        else:
            time.sleep(1) # attempting once every second
# Ok, I've got 'em. Let's iterate through each one

...

This will retry calling getUpdates every second for 30 seconds until the connection rights itself. You can tune the connection_timeout to be as big or small as needed in order to cover the intermittent connection.