DeadlineExceededError: ApplicationError: 5 in using urllib2.urlopen() function

Question

In my application I am using the urllib2.urlopen() function to call an api and get the result from that api.But this is not working fine.Sometimes it shows the result but sometimes it gives the following error:

Traceback (most recent call last):
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 700, in __call__
handler.get(*groups)
  File "/base/data/home/apps/s~malware-app/7.351334968546050391/main.py", line 505, in get
f = urllib2.urlopen('http://whoapi.com/api-v1/?domain=%s&rtype=alexarank&apikey=xyz'% domain)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 124, in urlopen
    return _opener.open(url, data)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 381, in open
response = self._open(req, data)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 399, in _open
  '_open', req)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 360, in _call_chain
result = func(*args)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 1114, in http_open
return self.do_open(httplib.HTTPConnection, req)
  File "/base/python_runtime/python_dist/lib/python2.5/urllib2.py", line 1087, in do_open
r = h.getresponse()
  File "/base/python_runtime/python_dist/lib/python2.5/httplib.py", line 197, in getresponse
self._allow_truncated, self._follow_redirects)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 260, in fetch
return rpc.get_result()
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 592, in get_result
return self.__get_result_hook(self)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 364, in _get_fetch_result
raise DeadlineExceededError(str(err))
DeadlineExceededError: ApplicationError: 5 

I saw try-except method for this but it dint work for my code. My code block :

 try:                      
   f = urllib2.urlopen('http://whoapi.com/api-v1/?domain=%s&rtype=serverip&apikey=xyzxyz'% domain)
   ip = f.read()
 except DeadlineExceededError, e:
   self.redirect('/error')

I am importing :

from google.appengine.runtime import DeadlineExceededError

From stackoverflow i got that its bcause server didn't responded in the specified time ,and we can handle the exception ..am but not able to do that. Any help would be appreciated. Thank you for your help

Answer 1

The default timeout for URL Fetch requests is just 5 seconds so you might want to increase it by using the urlfetch directly:

from google.appengine.api import urlfetch

try:
    resp = urlfetch.fetch('http://whoapi.com/api-v1/?domain=%s&rtype=serverip&apikey=xyzxyz'% domain, method=urlfetch.GET, deadline=10)
    ip = r.content
except urlfetch.DownloadError:
    self.redirect('/error')

If you consistently find exceeding it anyway, consider using asynchronous requests or moving the logic to task queue.