What's the purpose of AsyncContext.start(...) in Servlet 3.0?

Question

Servlet API says about "AsyncContext.start":

void start(java.lang.Runnable run)

Causes the container to dispatch a thread, possibly from a managed thread pool, to run the specified Runnable. The container may propagate appropriate contextual information to the Runnable.

From this description it's not clear how does it relate to task of optimizing thread usage when job requires waiting.

In "Servlet & JSP", Budi Kurniawan gives example of Servlet 3.0 async features, where he uses AsyncContext.start, I'll show simplified version of the example:

public void doGet(...) {
    final AsyncContext asyncContext = request.startAsync();

    asyncContext.start(new Runnable() {                        
        @ Override
        public void run() {
            // do some work here which involves waiting
            ...
            asyncContext.complete();
        }
    });
}

In most other examples I've met, the service method just stores the AsyncContext somewhere and it's processed somewhere else (eg. by a background thread). In this example it looks like the job is just passed to another thread, which completes the request. As I understand, now it's simply the worker thread, which wastes time on waiting.

Do you actually gain something by passing the job (which involves waiting) from one thread to another? If not, then what's the purpose of AsyncContext.start(...)?

Answer 1

You found a poor example, IMHO. In fact I wasn't even aware of AsyncContext.start() existence.

I had a quick look at how Jetty and Tomcat implement this. In fact, they seem to have some thread pool that handles asynchronous invocations independently.

Such usage of the API gives you nothing or very little. Instead of blocking the HTTP thread you are blocking some other thread pool. So I can imagine the application still accepts new connections, but the problem remains - the container can't handle them all because that extra thread pool is still limited.

The whole points of AsyncContext is the ability to handle more than one request by a single thread. Often you need only a single thread to handle thousands of asynchronous connections - e.g. when exactly one thread waits for data that is suppose to be broadcasted to several clients. Also see The Limited Usefulness of AsyncContext.start()

Answer 2

I had the same reaction at first -- if you're just passing the work to another thread, what do you gain? The spec is not much help in explaining why this is a good idea. But this post does an excellent job. Basically, it's to let the server degrade gracefully under heavy load rather than just fail by running out of threads. The actual work is done in a fixed-size pool of threads, so the server can accept any number of requests without having to keep a thread around for each one until it completes. Of course, you may have to tweak your O/S settings to be able to keep open thousands of sockets at a time.

Once you have this ability, you can more easily take advantage of the Comet (server push) architecture, where the client Javascript keeps an AJAX request open so that the server can notify it as soon as some event occurs, rather than having to poll the server to find out if something happened.