Does multi-threading improve performance if there are many requests at the same time?

Question

I am working on a Java web application using Spring 3 hosted on Tomcat 7 which needs to handle over 2.5k requests/second. I have a RequestProcesseor class to which process every HTTP requests with this method:

@Service
public class RequestProcesseor {

    public void processSomething(int value1, int value2) {
        // process something...
        // create and deep copy some object
        // some BigDecimal calculations
        // maybe some webservice calls
    }

}

There are over 2.5k requests at the same time and will call processSomething method. If I made this class multithreaded. Will it improve the performance? If yes, why? And how can I prove it?

The server has a 4-core CPU.

Answer 1

Even if you aren't doing any multithreading explicitly, your application server is implicitly dispatching each request to its own thread, so you already have concurrent load on the CPU.

Concurrent code will only help you if your request processing is CPU-limited, which is very rarely the case. Usually the bottleneck is the database or, more generally, the interface to other back-end subsystems.

If each request is processed by crunching a lot of in-memory data, and if the request-per-second load is low, then it can pay off to carefully divide the work among a few threads, no more than the actual CPU core count.

Therefore, since your server is quite heavily loaded, it will be almost certainly impossible to improve its performance by dispatching work to several threads. Note that it is dangerously easy to ruin the performance by multithreading.