Tomcat NIO connector with a blocking application

Question

After reading about the Tomcat NIO connector I still don't get one thing: is the nio connector beneficial if the application code is blocking, i.e. it blocks on reading from the database, on reading the file system, on calling external web services?

So, for example, you have a REST-like API that receives a request, reads something from the database, and returns a response. It doesn't use servlet 3 async, it just writes to the response.

I didn't find a full description of the thread pools used by the NIO connector, but I imagine it has a thread pool for handling the requests, so each request ends up in its own thread, which it can block.

If that's the case, are the benefits of NIO still there, or the blocking code diminishes the benefits of NIO (in terms of resource utilization)?

Answer 1

Is the nio connector beneficial if the application code is blocking...?

Yes, the NIO connector is built with the assumption that your app will block somewhere. The NIO connector basically has several socket placeholders and responds to new incoming requests until information starts getting written back.

I didn't find a full description of the thread pools used by the NIO connector

I think this is the start of your confusion. Tomcat NIO has a selector pool, not a thread pool (reference). The connector code polls each selector to see if it has incoming or outgoing bytes to send. In this sense, the selector for a given request will continue to receive information until there is enough to process the request with a Request/Response object that bridges the gap between synchronous I/O and asynchronous I/O (reference).

The polling code never blocks longer than the time it takes to serialize a packet of information, so it's free to handle new requests. The only real limitation is the amount of memory available to Tomcat. While there is a thread pool, the number of actual threads used are much lower than the number of connections the application can handle (reference).

While there are performance differences between Tomcat Connectors (reference), the difference in raw request/response time is pretty small when the servlet itself blocks. However, the difference in the number of simultaneous requests that Tomcat can handle is vastly different when you use non-blocking I/O.