HTTPException: Deadline exceeded while waiting for HTTP response from URL: #deadline

Question

We are using the developers python guide with Python data 2.15 library and as per example stated for app engine. createSite("test site one", description="test site one", source_site =("https://sites.google.com/feeds/site/googleappsforus.com/google-site-template" ))

We are getting an un-predictable response every time we use.

Exception: HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://sites.google.com/feeds/site/googleappsforyou.com

Did someone experience the same issue? Is it AppEngine or Sites API related?

Regards,

Answer 1

Deadline exceeded while waiting for HTTP response from URL is actually a DeadlineExceededError. When you are making a HTTP request, App Engine maps this request to URLFetch. URLFetch has its own deadline that is configurable. See the URLFetch documentation.

It seems that your client library catches DeadlineExceededError and throws HTTPException. Your client library either passes a deadline to URLFetch (which you would need to change) or the default deadline is insufficient for your request.

Try setting the default URLFetch deadline like this:

from google.appengine.api import urlfetch
urlfetch.set_default_fetch_deadline(45)

Also make sure to check out Dealing with DeadlineExceededErrors in the official Docs.

Keep in mind that any end-user initiated request must complete in 60 seconds or will encounter a DeadlineExceededError. See App Engine Python Runtime Request Timer.

Answer 2

The accepted solution did not work for me when working with the very recent versions of httplib2 and googleapiclient. The problem appears to be that httplib2.Http passes it's timeout argument all the way through to urlfetch. Since it has a default value of None, urlfetch sets the limit for that request to 5s irrespective of whatever default you set with urlfetch.set_default_fetch_deadline. It appears that you have a few options.

First option -- You can explicitly pass an instance of httplib2.Http around:

http = httplib2.Http(timeout=30)
...
client.method().execute(http=http)

Second option, you can set the default value using sockets¹:

import socket
socket.setdefaulttimeout(30)

Third option, you can tell appengine to use sockets for requests². To do that, you modify app.yaml:

env_variables:
   GAE_USE_SOCKETS_HTTPLIB : 'anyvalue'

^{1This might only work for paid apps since the socket api might not be present for unpaid apps...
2I'm almost certain this will only work for paid apps since the socket api won't be functional for unpaid apps...}