How does Go handle concurrent request on Google App Engine

Question

I'm a little confused on how Go handles concurrent requests on Google App Engine. So I'm hoping someone can provide some clarity.

Here are the facts that I have gathered:

1. Go is single threaded on App Engine. - this is because it is possible to do arbitrary pointer arithmetic by creating race conditions with multiple threads

2. Goroutines are multiplexed onto multiple OS threads so if one should block, such as while waiting for I/O, others continue to run.

3. [App Engine has a] 10 concurrent limit [which] is enforced through a limit on concurrent threads on every runtime. Most of such cases, our scheduler will try to spin up a new instance.

If Go is single threaded on App Engine then point 3 is moot. This leaves 1 and 2. If Go on App Engine is single threaded and threads are required to continue execution while blocking for I/O, then it seems that an App Engine Go instance will block all goroutines while waiting on I/O.

Is this correct? If not how does Go's concurrency really work on App Engine?

To help quantify things. If I were to hold a connection open for 30 seconds. How may concurrent connections could a single AE Go instance maintain?

Thank you.

EDIT: here's the feature request which will allow Go Instance to handle more then 10 concurrent request Allow configurable limit of concurrent requests per instance. Please star it.

Answer 1

A Go App Engine instance allows 10 concurrent requests, but only runs one CPU thread. In effect, multiple requests can be processed concurrently, but only one will be able to do CPU work at a time. If one request is, say, waiting for a datastore API call to return, another request is free to be processed by the same instance.

Your statement "If Go is single threaded on App Engine then point 3 is moot." is incorrect. There is still a limit of 10 concurrent in-flight requests to a single Go App Engine instance. The documentation is a bit loose with words when it talks about "threads".

Answer 2

I must admit I have no inside knowledge of AppEngine. This is all speculation and guessing, but I think it is somewhat reasonable.

Your app will never reach the ten thread limit. This is because there are very few reasons for threads to be created. First, the maximum number of goroutines running at once is set to one to enforce memory safety. Second, unlike a normal go program, app engine does not actually need to make syscalls. The only time it does is for networking. All IO in appengine can be muxed into one epoll thread. This means all you need is two threads at any given time. You can then add a third or forth thread in case every once in a while you need to run other syscalls such as allocating memory and accepting/closing connections. These are fast syscalls that block for a very small amount of time. Even with these other syscalls, you are still very far from the ten thread limit.

Concurrency is not affected because in the end, everything you do in appengine boils down to waiting for something over the network to return. You do not need multiple threads to be doing many things at once.