How to handle timeouts with httplib (python 2.6)?

Question

I'm using httplib to access an api over https and need to build in exception handling in the event that the api is down.

Here's an example connection:

connection = httplib.HTTPSConnection('non-existent-api.com', timeout=1)
connection.request('POST', '/request.api', xml, headers={'Content-Type': 'text/xml'})
response = connection.getresponse()

This should timeout, so I was expecting an exception to be raised, and response.read() just returns an empty string.

How can I know if there was a timeout? Even better, what's the best way to gracefully handle the problem of a 3rd-party api being down?

Answer 1

Even better, what's the best way to gracefully handle the problem of a 3rd-party api being down?

what's mean API is down , API return http 404 , 500 ...

or you mean when the API can't be reachable ?

first of all i don't think you can know if a web service in general is down before trying to access it so i will recommend for first one you can do like this:

import httplib

conn = httplib.HTTPConnection('www.google.com')  # I used here HTTP not HTTPS for simplify
conn.request('HEAD', '/')  # Just send a HTTP HEAD request 
res = conn.getresponse()

if res.status == 200:
   print "ok"
else:
   print "problem : the query returned %s because %s" % (res.status, res.reason)  

and for checking if the API is not reachable i think you will be better doing a try catch:

import httplib
import socket

try:
   # I don't think you need the timeout unless you want to also calculate the response time ...
   conn = httplib.HTTPSConnection('www.google.com') 
   conn.connect()
except (httplib.HTTPException, socket.error) as ex:
   print "Error: %s" % ex

You can mix the two ways if you want something more general ,Hope this will help

Answer 2

urllib and httplib don't expose timeout. You have to include socket and set the timeout there:

import socket
socket.settimeout(10) # or whatever timeout you want

Answer 3

This is what I found to be working correctly with httplib2. Posting it as it might still help someone :

    import httplib2, socket

    def check_url(url):
        h = httplib2.Http(timeout=0.1) #100 ms timeout
        try:
            resp = h.request(url, 'HEAD')
        except (httplib2.HttpLib2Error, socket.error) as ex:
            print "Request timed out for ", url
            return False
        return int(resp[0]['status']) < 400