What happens if an application calls more than 10 asynchronous URL Fetch on Google App Engine?

Question

Reading the Google App Engine documentation on asynchronous URL Fetch:

The app can have up to 10 simultaneous asynchronous URL Fetch calls

What happens if an application calls more than 10 async fetch at a time?
Does Google App Engine raise an exception or simply queue the remain calls waiting to serve them?

Answer 1

Umm, Swizzec is incorrect. Easy enough to test:

rpc = []
for i in range(1,20):
    rpc.append(urlfetch.createrpc())
    urlfetch.make_fetch_call(rpc[-1],"http://stackoverflow.com/questions/3639855/what-happens-if-i-call-more-than-10-asynchronous-url-fetch")

for r in rpc:
    response = r.get_result().status_code

This does not return any exceptions. In fact, this works just fine! Note that your results may vary for non-billable applications.

What Swizec is reporting is a different problem, related to maximum simultaneous connections INTO your application. For billable apps there is no practical limit here btw, it just scales out (subject to the 1000ms rule).

GAE has no way of knowing that your request handler will issue a blocking URL fetch, so the connection 500's he is seeing are not related to what his app is actually doing (that's an oversimplification btw, if your average request response time is > 1000ms your likelyhood of 500's increases).

Answer 2

This is an old question, but I believe the accepted answer is incorrect or outdated and may confuse people. It's been a couple of months that I actually tested this, but in my experience Swizec is quite right that GAE will not queue but rather fail most asynchronous URL fetches exceeding the limit of around 10 simultaneous ones per request.

See https://developers.google.com/appengine/docs/python/urlfetch/#Python_Making_requests and https://groups.google.com/forum/#!topic/google-appengine/EoYTmnDvg8U for a description of the limit.

David Underhill has come up with a URL Fetch Manager for Python, which queues asynchronous URL fetches that exceed the limit in application code.

I have implemented something similar for Java, which synchronously blocks (due to the lack of a callback function or ListenableFutures) additional requests:

/**
 * A URLFetchService wrapper that ensures that only 10 simultaneous asynchronous fetch requests are scheduled. If the
 * limit is reached, the fetchAsync operations will block until another request completes.
 */
public class BlockingURLFetchService implements URLFetchService {
    private final static int MAX_SIMULTANEOUS_ASYNC_REQUESTS = 10;

    private final URLFetchService urlFetchService = URLFetchServiceFactory.getURLFetchService();
    private final Queue<Future<HTTPResponse>> activeFetches = new LinkedList<>();

    @Override
    public HTTPResponse fetch(URL url) throws IOException {
        return urlFetchService.fetch(url);
    }

    @Override
    public HTTPResponse fetch(HTTPRequest request) throws IOException {
        return urlFetchService.fetch(request);
    }

    @Override
    public Future<HTTPResponse> fetchAsync(URL url) {
        block();

        Future<HTTPResponse> future = urlFetchService.fetchAsync(url);
        activeFetches.add(future);
        return future;
    }

    @Override
    public Future<HTTPResponse> fetchAsync(HTTPRequest request) {
        block();

        Future<HTTPResponse> future = urlFetchService.fetchAsync(request);
        activeFetches.add(future);
        return future;
    }

    private void block() {
        while (activeFetches.size() >= MAX_SIMULTANEOUS_ASYNC_REQUESTS) {
            // Max. simultaneous async requests reached; wait for one to complete
            Iterator<Future<HTTPResponse>> it = activeFetches.iterator();
            while (it.hasNext()) {
                if (it.next().isDone()) {
                    it.remove();
                    break;
                }
            }
        }
    }
}