Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What's the best way to parallelize a REST call?

I'm working on some java code, which processes multiple REST calls

call1()
call2()
call3()
...

I want to parallelize these calls, but perform my main code synchronously. I made a POC with lamba and a parallel stream:

List<Runnable> list = new ArrayList();
list.add(() -> {call1()});
list.add(() -> {call2()});
list.add(() -> {call3()});
list.add(...);

list.parallelStream()
            .forEach(Runnable::run);

Do you have another solution? I also checked to use async calls from Jersey client instead, but this would need more code changes.

like image 819
Bertrand Cedric Avatar asked Sep 20 '18 07:09

Bertrand Cedric


1 Answers

All you're looking for is to run your calls asynchronously. You can use CompletableFutures to submit the task, and then wait for them to complete:

list.stream() //you don't really need a parallel stream
    .map(CompletableFuture::runAsync)
    .collect(Collectors.toList()) //make sure all tasks are submitted
    .stream()
    .forEach(CompletableFuture::join);

This will submit all tasks (to run asynchronously), and then wait for each of them to finish running. When that happens, the method will return.

You may need to control the thread pool for your async tasks. This is an example using a 10-thread pool:

ExecutorService es = Executors.newFixedThreadPool(10);
list.stream()
    .map(r -> CompletableFuture.runAsync(r, es))
     ...
like image 55
ernest_k Avatar answered Sep 30 '22 12:09

ernest_k