How do you actually "manage" the max number of webthreads using Spring 5's Reactive Programming?

Question

When using a classical Tomcat approach, you can give your server a maximum number of threads it can use to handle web requests from users. Using the Reactive Programming paradigm, and Reactor in Spring 5, we are able to scale better vertically, making sure we are blocked minimally.

It seems to me that it makes this less manageable than the classical Tomcat approach, where you simply define the max number of concurrent requests. When you have a max number of concurrent requests, it's easier to estimate the maximum memory your application will need and scale accordingly. When you use Spring 5's Reactive Programming this seems like more of a hassle.

When I talk about these new technologies to sysadmin friends, they reply with worry about applications running out of RAM, or even threads on the OS level. So how can we deal with this better?

Answer 1

No blocking I/O at ALL

First of all, if you don't have any blocking operation then you should not worry at all about How much Thread should I provide for managing concurrency. In that case, we have only one worker which process all connections asynchronously and nonblockingly. And in that case, we may easily scale connection-servant workers which process all connections without contention and coherence (each worker has its own queue of received connections, each worker works on its own CPU) and we may scale application better in that case (shared nothing design).

Summary: in that case you manage max number of webthread identically as previously, by configuration application-container (Tomcat, WebSphere, etc) or similar in case of non-Servlet servers like Netty, or hybrid Undertow. The benefit - you may process muuuuuuch more users requests but with the same resources consumption.

Blocking Database and Non-Blocking Web API (such as WebFlux over Netty).

In case we should deal somehow with blocking I/O, for an instant communication with DB over blocking JDBC, the most appropriate way to keep your app scalable and efficient as possible we should use dedicated thread-pool for I/O.

Thread-pool requirements

First of all, we should create thread-pool with exactly the same amount of workers as available connections in JDBC connections-pool. Hence, we will have exactly the same amount of thread which will be blockingly wait for the response and we utilize our resources as efficiently as it possible, so no more memory will be consumed for Thread stack as it actually needed (In other word Thread per Connection model).

How to configure thread-pool accordingly to size of connection-pool

Since access to properties is varying for a particular database and JDBC driver, we may always externalize that configuration on a particular property, which in turn means that it may be configured by devops or sysadmin. A configuration of Threadpool (in our example it is configuring of Scheduler of Project Reactor 3) may looks like next:

@Configuration
public class ReactorJdbcSchedulerConfig {
    @Value("my.awasome.scheduler-size")
    int schedulerSize;

    @Bean
    public Scheduler jdbcScheduler() {
        return Schedulers.fromExecutor(new ForkJoinPool(schedulerSize));
        // similarly 
        // ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
        // taskExecutor.setCorePoolSize(schedulerSize);
        // taskExecutor.setMaxPoolSize(schedulerSize);
        // taskExecutor.setQueueCapacity(schedulerSize);
        // taskExecutor.initialize();
        // return Schedulres.fromExecutor(taskExecutor);
    }
}
...

    @Autowire
    Scheduler jdbcScheduler;


    public Mono myJdbcInteractionIsolated(String id) {
         return Mono.fromCallable(() -> jpaRepo.findById(id))
                    .subscribeOn(jdbcScheduler)
                    .publishOn(Schedulers.single());
    }
...

As it might be noted, with that technique, we may delegate our shared thread-pool configuration to an external team (sysadmins for an instance) and allows them to manage consumption of memory which is used for created Java Threads.

Keep your blocking I/O thread pool only for I/O work

This statement means that I/O thread should be only for operations which are blockingly waiting. In turn, it means that after the thread has done his awaiting the response, you should move result processing to another thread.

That is why in the above code-snippet I put .publishOn right after .subscribeOn.

So, to summarize, with that technique we may allow external team managing application sizing by controlling thread-pool size to connection-pool size accordingly. All results processing will be executed within one thread and there will be no redundant, uncontrolled memory consumption hence.

Finally, Blocking API (Spring MVC) and blocking I/O (Database access)

In that case, there is no need for reactive paradigm at all since you don't get any profit from that. First of all, Reactive Programming requires particular mind shifting, especially in the understanding of the usage of functional techniques with Reactive libraries such as RxJava or Project Reactor. In turn for non-prepared users, it gives more complexity and causes more "What ****** is going on here???". So, in case of blocking operations from both ends, you should think twice do you really need Reactive Programming here.

Also, there is no magic for free. Reactive Extensions comes with a lot of internal complexity and using all that magical .map, .flatMap, etc., you may lose in overall performance and memory consumption instead of winning like in case of end-to-end non-blocking, async communication.

That means that old good imperative programming will be more suitable here and it will much easier to control your application sizing in memory using old good Tomcat configuration management.