Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parallel stream doesn't set Thread.contextClassLoader after tomcat upgrade

After tomcat upgrade from 8.5.6 to 8.5.28 parallel stream stopped supplying Threads with contextClassLoader:

Because of it Warmer::run can't load classes in it.

warmers.parallelStream().forEach(Warmer::run);

Do you have any ideas what Tomcat was supplying for contextClassLoaders for new Threads?

ParallelStream uses ForkJoinPool in newest Tomcat.

like image 862
oneat Avatar asked Mar 05 '18 12:03

oneat


People also ask

How do you control threads in a parallel stream?

The idea is to create a custom fork-join pool with a desirable number of threads and execute the parallel stream within it. This allows developers to control the threads that parallel stream uses. Additionally, it separates the parallel stream thread pool from the application pool which is considered a good practice.

Does parallel stream use thread pool?

2. Parallel Stream. The default processing that occurs in such a Stream uses the ForkJoinPool. commonPool(), a thread pool shared by the entire application.

When we should not use parallel stream?

Similarly, don't use parallel if the stream is ordered and has much more elements than you want to process, e.g. This may run much longer because the parallel threads may work on plenty of number ranges instead of the crucial one 0-100, causing this to take very long time.

What does stream parallel do?

Parallel streams enable us to execute code in parallel on separate cores. The final result is the combination of each individual outcome.


2 Answers

Common ForkJoin pool is problematic and could be responsible for memory leaks and for applications being able to load classes and resources from other contexts/applications (potential security leak if your tomcat is multi tenant). See this Tomcat Bugzilla Report.

In Tomcat 8.5.11 they had applied fix to the above issues by introducing SafeForkJoinWorkerThreadFactory.java

In order for your code to work, you can do the following, which will supply explicit ForkJoin and its worker thread factory to the Stream.parallel() execution.

ForkJoinPool forkJoinPool = new ForkJoinPool(NO_OF_WORKERS);
forkJoinPool.execute(() -> warmers.parallelStream().forEach(Warmer::run));
like image 50
diginoise Avatar answered Sep 19 '22 16:09

diginoise


This saved my day! I had to do it like this to make it work:

private static class CustomForkJoinWorkerThread extends ForkJoinWorkerThread {
    CustomForkJoinWorkerThread(ForkJoinPool pool) {
        super(pool);
        setContextClassLoader(Thread.currentThread().getContextClassLoader());
    }
}

private ForkJoinPool createForkJoinPool() {
    return new ForkJoinPool(
            ForkJoinPool.getCommonPoolParallelism(),
            CustomForkJoinWorkerThread::new,
            null,
            false
    );
}


createForkJoinPool().submit(() -> stuff.parallelStream().doStuff())
like image 38
daasis Avatar answered Sep 21 '22 16:09

daasis