Does the code with CompletableFutures and no custom Executors use only the number of threads equal to the number of cores?

Question

I am reading java 8 in action, chapter 11 (about CompletableFutures), and it got me thinking about my company's code base.

The java 8 in action book says that if you have code like I write down below, you will only use 4 CompletableFutures at a time(if you have a 4 core computer). That means that if you want to perform for example 10 operations asynchronously, you will first run the first 4 CompletableFutures, then the second 4, and then the 2 remaining ones, because the default ForkJoinPool.commonPool() only provides the number of threads equal to Runtime.getRuntime().availableProcessors().

In my company's code base, there are @Service classes called AsyncHelpers, that contain a method load(), that uses CompletableFutures to load information about a product asynchronously in separate chunks. I was wondering if they only use 4 threads at a time.

There are several such async helpers in my company's code base, for example there's one for product list page (PLP) and one for product details page(PDP). A product details page is a page dedicated to a specific product showing it's detailed characteristics, cross-sell products, similar products and many more things.

There was an architectural decision to load the details of the pdp page in chunks. The loading is supposed to happen asynchronously, and the current code uses CompletableFutures. Let's look at pseudocode:

static PdpDto load(String productId) {
    CompletableFuture<Details> photoFuture =
            CompletableFuture.supplyAsync(() -> loadPhotoDetails(productId));
    CompletableFuture<Details> characteristicsFuture =
            CompletableFuture.supplyAsync(() -> loadCharacteristics(productId));
    CompletableFuture<Details> variations =
            CompletableFuture.supplyAsync(() -> loadVariations(productId));

    // ... many more futures

    try {
        return new PdpDto( // construct Dto that will combine all Details objects into one
                photoFuture.get(),
                characteristicsFuture.get(),
                variations.get(),
                // .. many more future.get()s
        );
    } catch (ExecutionException|InterruptedException e) {
        return new PdpDto(); // something went wrong, return an empty DTO
    }
}

As you can see, the code above uses no custom executors.

Does this mean that if that load method has 10 CompletableFutures and there are currently 2 people loading the PDP page, and we have 20 CompletableFutures to load in total, then all those 20 CompletableFutures won't be executed all at once, but only 4 at a time?

My colleague told me that each user will get 4 threads, but I think the JavaDoc quite clearly states this:

public static ForkJoinPool commonPool() Returns the common pool instance. This pool is statically constructed; its run state is unaffected by attempts to shutdown() or shutdownNow(). However this pool and any ongoing processing are automatically terminated upon program System.exit(int). Any program that relies on asynchronous task processing to complete before program termination should invoke commonPool().awaitQuiescence, before exit.

Which means that there's only 1 pool with 4 threads for all users of our website.

Answer 1

Yes, but it’s worse than that...

The default size of the common pool is 1 less than the number of processors/cores (or 1 if there’s only 1 processor), so you’re actually processing 3 at a time, not 4.

But your biggest performance hit is with parallel streams (if you use them), because they use the common pool too. Streams are meant to be used for super fast processing, so you don’t want them to share their resources with heavy tasks.

If you have task that is designed to be asynch (ie take more than a few milliseconds) then you should create a pool to run them in. Such a pool can be statically created and reused by all calling threads, which avoids overhead of pool creation per use. You should also tune the pool size by stress testing your code to find the optimum size to maximise throughput and minimise response time.