Tomcat 7 Async Processing failing - only one request processed simultanously

Question

I was trying to implement COMET chat using Async Processing defined in Servlet API 3. It was not working - chat got blocked, so I have created debug servlet to test async part only.

This is my doGet method:

@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    log.debug("doGet called");
    int timeout = 30 + RandomUtils.nextInt(60);
    String message = RandomStringUtils.randomAlphanumeric(50 + RandomUtils.nextInt(250));
    response.setHeader("Access-Control-Allow-Origin", "*");
    final AsyncContext context = request.startAsync();

    synchronized(items) {
        items.add(new RequestItem(context, message, timeout));
    }
    log.debug("doGet created request and finished");
}

I'm putting request items in queue, and there's a thread running, that will take the items after specified timeout and send response to AsyncContext, printing message about it. The problem is, the thread is blocked until the AsyncContext gets responded. This is what is visible in my log after requesting 4 page loads in browser:

2011-12-08 13:56:36,923 DEBUG [my.servlet.TestAsyncServlet] doGet called
2011-12-08 13:56:36,952 DEBUG [my.servlet.TestAsyncServlet] doGet created request and finished
2011-12-08 13:57:39,934 TRACE [my.servlet.TestAsyncServlet] respond on item RequestItem [context=org.apache.catalina.core.AsyncContextImpl@175870a, message=zEQpATavzwFl6qIbBKve4OzIY9UUuZBwbqN1TC5KpU3i8LM9B6ChgUqaRmcT2yF, timeout=0]
2011-12-08 13:57:39,962 DEBUG [my.servlet.TestAsyncServlet] doGet called
2011-12-08 13:57:39,962 DEBUG [my.servlet.TestAsyncServlet] doGet created request and finished
2011-12-08 13:58:53,949 TRACE [my.servlet.TestAsyncServlet] respond on item RequestItem [context=org.apache.catalina.core.AsyncContextImpl@88ee03, message=pKHKC632CPIk7hGLV0YqCbQl1qpWIoyNv5OWCp21bEqoni1gbY79HT61QEUS2eCjeTMoNEwdqKzCZNGgDngULysSzVdzFTnQQ5cQ8JvcYnp1pLVqGTueJPWnbRdUuO, timeout=0]
2011-12-08 13:58:53,960 DEBUG [my.servlet.TestAsyncServlet] doGet called
2011-12-08 13:58:53,960 DEBUG [my.servlet.TestAsyncServlet] doGet created request and finished
2011-12-08 13:59:36,954 TRACE [my.servlet.TestAsyncServlet] respond on item RequestItem [context=org.apache.catalina.core.AsyncContextImpl@197950e, message=43FPeEUZWBLqgkAqS3WOFMiHUMVvx6o4jNqWLx8kUvwxqJqpOZyGCtiIcr7yw, timeout=0]
2011-12-08 13:59:36,999 DEBUG [my.servlet.TestAsyncServlet] doGet called
2011-12-08 13:59:36,999 DEBUG [my.servlet.TestAsyncServlet] doGet created request and finished
2011-12-08 14:00:34,957 TRACE [my.servlet.TestAsyncServlet] respond on item RequestItem [context=org.apache.catalina.core.AsyncContextImpl@1cb1278, message=r69Y4NQsyR1vj0kzUlHssic2x1Yrr6T09IGKjWAH1E6Lz4VhFTy9dQHi5CPeTObyjLLBDlCLEDfiyMUnVkVIEgYG7r47Ak4w30RklhzdEi9nthqdfNkry6nyjircsFPX534NqWjI1LwsrGq5nOa3ZYtfjfPVpGlk4KDmWP11L53YntO3GmptZPKa50gcqj9i, timeout=0]

As it is to see, the next doGet method is called only after previous request is (theoretically asynchronous) answered. So the whole async thing is not working at all! And here's the web.xml declaration:

  <servlet>
    <servlet-name>TestAsyncServlet</servlet-name>
    <servlet-class>my.servlet.TestAsyncServlet</servlet-class>
    <async-supported>true</async-supported>
  </servlet>
  <servlet-mapping>
    <servlet-name>TestAsyncServlet</servlet-name>
    <url-pattern>/test-async</url-pattern>
  </servlet-mapping>

I'm doing all as to be found in Internet. I don't see the place of mistake to be made. I've found nothing special to configurure in servlet.xml. So the question is, why it is not working as it should?

Answer 1

OK, as the part of the research I've written testing programm that opened multiply connections to tomcat and did GET/POST on the async servlet. I've debugged and rechecked my server.xml configuration, limited thread pool for better visibility of test results etc. Now my configuration of connector looks like that:

<Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol" 
               minProcessors="3"
               maxProcessors="8"
               maxThreads="20"
               connectionTimeout="150000" 
               asyncTimeout="150000" />

And this is working! I've made test using NIO and made 1000 connections at one time, and all of them got processed in one time.

However, the effect I've described, still exists in browser. When I try to load servlet on 10 tabs, first get loaded, than second etc. It seems to be browser's behaviour, nothing gets blocked on server. When I opened 3 browsers (Firefox, Chrome, Opera) I've got 3 connections processed on one time.

So, the asynchronous processing defined in Servlet API 3.0 works on Tomcat 7, however, it must be tested with own programm not with multiple tabs in browser... testing COMET chat in multiple tabs can also not work as expected. However, in real-life example one computer will open only one connection. And, the browser's behaviour is not any server's fault.

edit After Spring MVC solution was included into web application, async processing mode was stopped (parameter from web.xml was ignored). However, the async support can be set up manually via adding line:

request.setAttribute("org.apache.catalina.ASYNC_SUPPORTED", true);

Answer 2

I think you have your server configured correctly. If you're just loading with a browser then the problem may be the way in which your browser is working. If you just access the web page with the browser then the browser will try and load the entire page and block until that is finished. Your browser won't finish the load until the async request is finished.

For example, if you were to look at the messaging with a tool like Fiddler I'd expect you'd see the following (assuming delay is 4):

Browser -> Server [Time: 0]   Request
Server -> Browser [Time: 0.1] Async Response
Server -> Browser [Timer: 4]  Complete Response
Browser shows page loaded.

If you took out the async mode you'd see:

Browser -> Server [Time: 0] Request
Server -> Browser [Time: 4] Response
Browser shows page loaded.

In both examples the page won't be fully loaded until the whole request is finished, even though one is asynchronous and one is synchronous. To fully take advantage of asynchronous requests you're going to need a smarter client, e.g. Javascript or flex or something.

In other words, I don't think you can say the server is correct or not by just loading with the browser. I would grab a tool like Fiddler and see exactly what HTTP messages are going across.