How to set limit to the number of concurrent request in servlet?

Question

I got this servlet which return a pdf file to the client web browser. We do not want to risk any chance that when the number of request is too much, the server is paralyzed.

We would like to make an application level (program) way to set a limit in the number of concurrent request, and return a error message to the browser when the limit is reached. We need to do it in applicantion level because we have different servlet container in development level(tomcat) and production level(websphere).

I must emphasize that I want to control the maximum number of request instead of session. A user can send multiple request over the server with the same session.

Any idea? I've thought about using a static counter to keep track of the number of request, but it would raise a problem of race condition.

Answer 1

I'd suggest writing a simple servlet Filter. Configure it in your web.xml to apply to the path that you want to limit the number of concurrent requests. The code would look something like this:

public class LimitFilter implements Filter {
    private int limit = 5;
    private int count;
    private Object lock = new Object();

    public void doFilter(ServletRequest request, ServletResponse response,
            FilterChain chain) throws IOException, ServletException {
        try {
            boolean ok;
            synchronized (lock) {
                ok = count++ < limit;
            }
            if (ok) {
                // let the request through and process as usual
                chain.doFilter(request, response);
            } else {
                // handle limit case, e.g. return status code 429 (Too Many Requests)
                // see https://www.rfc-editor.org/rfc/rfc6585#page-3
            }
        } finally {
            synchronized (lock) {
                count--;
            }           
        }
    }
}

Or alternatively you could just put this logic into your HttpServlet. It's just a bit cleaner and more reusable as a Filter. You might want to make the limit configurable through the web.xml rather than hard coding it.

Ref.:
Check definition of HTTP status code 429.

Answer 2

You can use RateLimiter. See this article for explanation.