Is using Servlet 3.0 async redundant when deployed to Tomcat with the NIO connector?

Question

I saw this thread: Java - Async in Servlet 3.0 vs NIO in Servlet 3.1 but it seems to be concerning the Servlet 3.1 NIO (as opposed to the Tomcat NIO HTTP connector).

It's my understanding that configuring Tomcat with the NIO HTTP connectors implementation (the default in Tomcat 8 and above) the actual work to process the request and produce a response is done on a separate worker thread, and the poller threads that read/write the data remain unblocked.

This seems to be the same problem that Async Servlet 3.0 solves, in that the work done on the request/response is done on a worker thread independent of the http connection threads.

So are they two solutions to the same problem? In other words, is there any benefit to writing my code in an async manner if the servlet container is already asynchronous?

Answer 1

It is easier to understand this by understanding the potential places where IO happens at different points in the request processing from the container to the application code. The container connector ( BIO / NIO) job is to accept the socket connection and hand it over to thread which at some point of time invokes the Servlet GET/POST method. Now Tomcat NIO connector is basically container's decision to use the Java NIO facility ( Selector / Channel ) to handle multiple IO channels with less threads. Selector provides a mechanism for monitoring one or more NIO channels and recognizing when one or more become available for data transfer , so with selector container can use one thread instead of several to manage multiple channels. These ready channels are then served by threads which are potentially less in number than required in BIO connector.

As an aside - there are OS level optimizations which are done at this level to improve NIO functioning. For e.g., Java ships with a java.nio.channels.SelectorProvider implementation that is based on the Linux epoll event notification facility. The epoll facility is available in the Linux 2.6, and newer, kernels. The new epoll-based SelectorProvider implementation is more scalable than the traditional poll-based SelectorProvider implementation when there are thousands of SelectableChannels registered with a Selector. The new SelectorProvider implementation will be used by default when the 2.6 kernel is detected. The poll-based SelectorProvider will be used when a pre-2.6 kernel is detected.

Now coming back to the problem at hand. Prior to Servlet 3.0 the entire servlet processing was synchronous which was improved in Servet 3.0 so that the GET/POST method now returns immediately but without writing to response unless it is deemed complete by a call to AsyncContext complete. So far so good. But there was another problem. The Servlet processing can potentially include reading/writing to input stream/output stream available from request. The nature of this IO was still traditional. Servlet 3.0 allowed asynchronous request processing but the IO read/write were still old style for e.g., while reading a big request payload from a client with slow speed the thread would block waiting for data - so Servlet 3.1 Non-blocking IO to the rescue where now you can have a callback style invocation of your reading/writing logic whenever the data is ready instead of the thread waiting for it. How this works in code can be seen here.

We still have to remember the database IO ( if any) is still again traditional waiting IO. That area is controlled by database drivers who adhere to JDBC API and who knows someday they provide API for non-blocking IO as well. Oracle is already leading one such initiative Asynchronous Database Access which also looks like to be utilizing Java NIO. This will pave way for making the entire request processing non-blocking.

Also there is another promising work being done by Spring - Reactive relational database connectivity that aims to solve this issue.