Springboot webflux reactor concurrency model

Question

I'd like to understand more about the underlying concurrency model for the springboot webflux?

For CPU intense webservice, is the traditonal Blocking Multithreaded model a better fit? Or is it in general traditional thread pool model a better fit according to this paper https://people.eecs.berkeley.edu/~brewer/papers/threads-hotos-2003.pdf ?

If I have a blocking step in the reactor chain, I use the publishOn to schedule it to a different threadpool. Does that free up the original thread and make the whole chain still nonblocking?

Answer 1

Where WebFlux fits well and how it works under the hood

From my standpoint, the best fit for Spring WebFlux is network intensive applications. Under the hood, Spring WebFlux uses Reactor-Netty which is a backpressure-supported wrapper around Netty. Reactor-Netty uses almost the same Netty Threading strategy and creates Runtime.getRuntime().availableProcessors() * 2 Threads for its internal EventLoopThreadPool. It means that each Thread is ~ bound to its own core/processor and work with minimal contention for CPU resource. Such model works very well with end-to-end non-blocking communication. So in the case incoming request end up with a remote network call, that invocation should be executed in the non-blocking fashion so the same thread could be back to the rest of the tasks in the even-loop queue. Such a technique allows us to efficiently utilize our hardware with almost no overhead spent on context-switching and high contention, which is a common property of blocking communication with a high number of involved threads.

The role of Project Reactor

The central role of Project Reactor in Spring WebFlux is to provide a programming model that preserve clarity of the complex non-blocking, asynchronous execution. It hides the complexity of continuation of data processing and allows us easily build a functional, declarative pipe of elements processing. Also, the Reactor's threading model is just a couple of operators that enable sophisticated and efficient elements processing rescheduling on a dedicated thread-pool with no headache. Under the hood, are used the same ThreadPools and ExecutorServices taken from Java Core library.

How to deal with CPU intensive tasks, does Project Reactor fits well there?

I would say - Reactor Netty fits well for CPU intensive task as well. But in that case, the Project Reactor should be used appropriately. In case of the complex algorithm processing or similar work, it is better to use pure Java since Reactor adds around 100-150% overhead regarding performance.

How to build efficient CPU intensive task processing with Project Reactor and Reactor-Netty (WebFlux)

I would recommend following pattern "Queue of Work" so each thread will take a new task once the previous is completed.

In case we have a CPU intensive task it is always recommended to schedule it on the dedicated Thread Pool. Even though it will add a little bit of overhead, we will have higher latency for I/O reading writing which is an integral part of any networked app. In the case of Netty, we will be sure that Netty's EventLoop does only reading and writing to the network and nothing more.

To schedule tasks on the dedicated thread-pool, we may follow the technique shown in the code sample below:

@PostMapping
public Mono<CpuIntensiveResult> cpuIntensiveProcessingHandler(
    Mono<CpuIntensiveInput> monoInput
) {
    return monoInput
        .publishOn(Schedulers.fromExecutorService(myOwnDedicatedExecutor))
        .map(i -> doCpuIntensiveInImperativeStyle(i));
}

As we can see from the code above, using one operator from the Project Reactor arsenal, we can easily schedule work processing on a dedicated Threadpool. In turn, we can quickly wrap any existing Executor Service into Scheduler and use whit-in Reactor ecosystem

What about blocking over multithreaded technique?

With usual, the multithreaded technique where we have more threads than cores, we will not get any benefits. The drawback here is the same - context-switching and contention. It would seem that the tasks are processed simultaneously. However, a system scheduler does the same hard work of CPU time allocation between concurrent Threads. It will share the same CPU between a couple of intensive tasks, so it results in the end in a higher latency for each task. On average the processing time will be higher than the same work is done by the same number of Thread as CPUs/Cores in the system.

A couple of notes regarding Threading models as a "true" processing model.

According to the mentioned white-paper, the Threading model is a true programming model. I guess, the authors of this paper is talking about Green Threads. Green Threads could be better programming model, but in the end, you will have to follow the same rules mentioned above for Reactor programming model. The drawback of Thread and subsequently imperative programming model is an unability to work with the stream of data, where Reactor programming model fits super well.

Also, I would recommend to revisit Universal Scalability Law and review the problematic of contention and coherence (which is relevant to current Java Threading executions). Also, a good overview of scalability is explained in the following paper.

Another sample of efficient usage of asynchronous + non-blocking request processing is Facebook architecture, which at pick-load transforms a queue of work to a stack of work which allows preserving lowest latency.