Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a limit on the number of asynchronous urlfetch calls I can run simultaneously?

I noticed what appears to be a limit on simultaneous asynchronous calls of urlfetch in the Java implementation (as noted here: http://code.google.com/appengine/docs/java/urlfetch/overview.html)

but not in the python documentation:

http://code.google.com/appengine/docs/python/urlfetch/asynchronousrequests.html

So is it the case that the python version of async urlfetch also has an upper limit of 10 and it's just not documented (or documented elsewhere)? Or is the limit something else (or non-existant)?

like image 727
ryan Avatar asked Jun 30 '10 03:06

ryan


2 Answers

The limit for Python is just not documented in that page but in another one, which says (in the middle of the last paragraph of this section):

The app can have up to 10 simultaneous asynchronous URL Fetch calls.

As you see, that's the same limit as for Java.

like image 58
Alex Martelli Avatar answered Nov 04 '22 17:11

Alex Martelli


umm - that may be true for non-billable apps, but try this in a billable app:

from google.appengine.api import urlfetch
rpc = []
for x in range(1,30):
   rpc.append(urlfetch.create_rpc())
   urlfetch.make_fetch_call(rpc[-1],"http://stackoverflow.com/questions/3639855/what-happens-if-i-call-more-than-10-asynchronous-url-fetch")

for r in rpc:
   response = r.get_result()
   logging.info("Response: %s", str(response.status_code))

It just works... So the limit for billable apps is in fact higher (but isn't documented!)

like image 39
Cloudbreak NZ Avatar answered Nov 04 '22 16:11

Cloudbreak NZ