Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tomcat 7 Async Processing

I am wondering how Tomcat 7 implements async processing. I understand that the request thread returns immediately, allowing the request thread to immediately listen for a new request and respond to it.

How is the 'async' request handled? Is there a separate thread pool that handles async requests? I assume blocking IO is handled using something like java.nio.Selector for performance. What about threads that are blocking on CPU calculations?

like image 588
John Cheng Avatar asked Sep 02 '11 17:09

John Cheng


People also ask

Is Tomcat asynchronous?

In case of Tomcat the latter approach uses Tomcat's thread pool defined in server. xml . Non-blocking IO (NIO); Although it is asynchronous as well it is a different story.

Can Tomcat serve static files?

Tomcat will serve any static content from a WAR file using the DefaultServlet.

Does Tomcat support non-blocking?

However, Tomcat also supports Non-Blocking I/O. It is said that it supports near 13,000 simultaneous requests in NIO mode. Still, there is a limit. The acceptCount attribute defines the maximum number of HTTP connections that will be put in a queue while there is no free connection available to serve.

What is maxThreads in Tomcat?

By default, Tomcat sets maxThreads to 200, which represents the maximum number of threads allowed to run at any given time. You can also specify values for the following parameters: minSpareThreads : the minimum number of threads that should be running at all times.


1 Answers

You are mixing up different concepts. You must distinguish between:

  1. Asynchronous request handing as per Servlet 3.0; an API that enables you to decouple an incoming servlet request from the web container thread pool. It does not create any threads on the fly. It's on the user of the interface to implement a proper multi-threaded solution. It does not relate to non-blocking IO.
  2. Thread pool; provides a mechanism to obtain and manage threads in a web container. When it comes to asynchronous request handing you have 2 options. You can define your own ExecutorService and use it to further process the request or you can create a new Runnable and submit it to the obtained AsyncContext by calling AsyncContext.start(). In case of Tomcat the latter approach uses Tomcat's thread pool defined in server.xml.
  3. Non-blocking IO (NIO); Although it is asynchronous as well it is a different story. It relates to non-blocking IO operations like disk or network IO. If you want to enable NIO for HTTP request processing, have a look at Tomcat's documentation.

The below example outlines how it can work. It uses only one thread for worker jobs. If you run it from 2 different browsers in parallel the output looks like this (I use a custom logger):

   DATE                         THREAD_ID  LEVEL      MESSAGE
2011-09-03 11:51:22.198 +0200      26        I:     >doGet: chrome
2011-09-03 11:51:22.204 +0200      26        I:     <doGet: chrome
2011-09-03 11:51:22.204 +0200      28        I:     >run: chrome
2011-09-03 11:51:27.908 +0200      29        I:     >doGet: firefox
2011-09-03 11:51:27.908 +0200      29        I:     <doGet: firefox
2011-09-03 11:51:32.227 +0200      28        I:     <run: chrome
2011-09-03 11:51:32.228 +0200      28        I:     >run: firefox
2011-09-03 11:51:42.244 +0200      28        I:     <run: firefox

You see that the doGet methods immediately finish, whereas the worker still runs. The 2 test requests: http://localhost:8080/pc/TestServlet?name=chrome and http://localhost:8080/pc/TestServlet?name=firefox.

Simple example Servlet

@WebServlet(asyncSupported = true, value = "/TestServlet", loadOnStartup = 1)
public class TestServlet extends HttpServlet {
    private static final Logger LOG = Logger.getLogger(TestServlet.class.getName());
    private static final long serialVersionUID = 1L;
    private static final int NUM_WORKER_THREADS = 1;

    private ExecutorService executor = null;

    @Override
    public void init() throws ServletException {
        this.executor = Executors.newFixedThreadPool(NUM_WORKER_THREADS);
    }

    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {

        final String name = request.getParameter("name");
        LOG.info(">doGet: " + name);

        AsyncContext ac = request.startAsync(); // obtain async context
        ac.setTimeout(0); // test only, no timeout

        /* Create a worker */
        Runnable worker = new TestWorker(name, ac);

        /* use your own executor service to execute a worker thread (TestWorker) */
        this.executorService.execute(worker);

        /* OR delegate to the container */
        // ac.start(worker);

        LOG.info("<doGet: " + name);
    }
}

...and the TestWorker

public class TestWorker implements Runnable {
    private static final Logger LOG = Logger.getLogger(TestWorker.class.getName());
    private final String name;
    private final AsyncContext context;
    private final Date queued;

    public TestWorker(String name, AsyncContext context) {
        this.name = name;
        this.context = context;
        this.queued = new Date(System.currentTimeMillis());
    }

    @Override
    public void run() {

        LOG.info(">run: " + name);

        /* do some work for 10 sec */
        for (int i = 0; i < 100; i++) {
            try {
                Thread.sleep(100);
            } catch (InterruptedException e) {
                throw new RuntimeException(e);
            }
        }

        ServletResponse response = this.context.getResponse();
        response.setContentType("text/plain");

        try {
            PrintWriter out = response.getWriter();
            out.println("Name:\t\t" + this.name);
            out.println("Queued:\t\t" + this.queued);
            out.println("End:\t\t" + new Date(System.currentTimeMillis()));
            out.println("Thread:\t\t" + Thread.currentThread().getId());
            out.flush();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }

        this.context.complete();

        LOG.info("<run: " + name);
    }
}
like image 70
home Avatar answered Oct 20 '22 06:10

home