Combining multiple CompletableFutures

Question

I have the following component:

private JobInfo aggregateJobInfo() {
    final JobsResult jobsResult = restClient().getJobs();
    final List<String> jobIds = extractJobIds(jobsResult);

    //fetch details, exceptions and config for each job
    final List<JobDetails> jobDetails = jobIds.stream().map(jobId -> {
        final JobDetailResult jobDetailResult = restClient().getJobDetails(jobId);
        final JobExceptionsResult jobExceptionsResult = restClient().getJobExceptions(jobId);
        final JobConfigResult jobConfigResult = restClient().getJobConfig(jobId);
        return new JobDetails(jobDetailResult, jobExceptionsResult, jobConfigResult);
    }).collect(Collectors.toList());
    return new JobInfo(jobsResult, jobDetails);
}

private static List<String> extractJobIds(final JobsResult jobsResult) {
    final ArrayList<String> jobIds = new ArrayList<>();
    jobIds.addAll(jobsResult.getRunning());
    jobIds.addAll(jobsResult.getFinished());
    jobIds.addAll(jobsResult.getCanceled());
    jobIds.addAll(jobsResult.getFailed());
    return jobIds;
}

It just calls some ENDPOINTS and aggergates some data. Now I'm trying to make that non-blocking by using CompletableFutures, which I didn't really used before..

private CompletableFuture<JobInfo> aggregateJobInfo() {
    final CompletableFuture<JobsResult> jobsResultFuture = restClient().getJobs();
    final CompletableFuture<List<String>> jobIdsFuture = jobsResultFuture.thenApply(JobInfoCollector::extractJobIds);

     //fetch details, exceptions and config for each job
    final CompletableFuture<List<CompletableFuture<JobDetails>>> jobDetailsFuture = jobIdsFuture.thenApply(jobIds -> {
        return jobIds.stream().map(jobId -> {
            final CompletableFuture<JobDetailResult> jobDetailsResultFuture = restClient().getJobDetails(jobId);
            final CompletableFuture<JobExceptionsResult> jobExceptionsFuture = restClient().getJobExceptions(jobId);
            final CompletableFuture<JobConfigResult> jobConfigFuture = restClient().getJobConfig(jobId);
            return jobDetailsResultFuture.thenCompose(jobDetailResult -> {
                return jobExceptionsFuture.thenCombine(jobConfigFuture, (jobExceptionsResult, jobConfigResult) -> {
                    return new JobDetails(jobDetailResult, jobExceptionsResult, jobConfigResult);
                });
            });

        }).collect(Collectors.toList());
    });
    return null;

My problem is how to create CompletableFuture here when JobInfo is `new JobInfo(jobsResult, jobDetails)?!

As I said, I'm new to this, maybe my approach is bad and there are better solutions?

Any ideas appreciated, thanks in

First working version:

private CompletableFuture<JobInfo> aggregateJobInfo() {

    final CompletableFuture<JobsResult> jobsResultFuture = restClient().getJobs();
    final CompletableFuture<List<String>> jobIdsFuture = jobsResultFuture.thenApply(JobInfoCollector::extractJobIds);

    final CompletableFuture<List<CompletableFuture<JobDetails>>> jobDetailsFutureListFuture =
            jobIdsFuture.thenApply(jobIds -> jobIds.stream().map(jobId -> {
                final CompletableFuture<JobDetailResult> jobDetailsResultFuture = restClient().getJobDetails(jobId);
                final CompletableFuture<JobExceptionsResult> jobExceptionsFuture = restClient().getJobExceptions(jobId);
                final CompletableFuture<JobConfigResult> jobConfigFuture = restClient().getJobConfig(jobId);
                return jobDetailsResultFuture.thenCompose(jobDetailResult ->
                        jobExceptionsFuture.thenCombine(jobConfigFuture, (jobExceptionsResult, jobConfigResult) ->
                                new JobDetails(jobDetailResult, jobExceptionsResult, jobConfigResult)));
            }).collect(Collectors.toList()));

    return jobDetailsFutureListFuture.thenCompose(jobDetailsFutures ->
            CompletableFuture.allOf(jobDetailsFutures.toArray(
                    new CompletableFuture[jobDetailsFutures.size()])).thenApply(aVoid ->
                    jobDetailsFutures.stream()
                            .map(CompletableFuture::join)
                            .collect(Collectors.toList())))
            .thenApply(jobDetails -> jobsResultFuture.thenApply(jobsResult ->
                    new JobInfo(jobsResult, jobDetails)))
            .join();
}

Answer 1

You have:

• CompletableFuture<JobsResult> jobsResultFuture
• CompletableFuture<List<CompletableFuture<JobDetails>>> jobDetailsFuture
• JobInfo(JobsResult a, List<JobDetails> b)

You want

CompletableFuture<JobInfo>

additional observation: jobDetailsFuture can only be completed when jobsResultFuture has completed.

So you can implement the following:

1. List<CompletableFuture<JobDetails>> -> Void via allOf in thenCompose
2. Void + List<CompletableFuture<JobDetails>> (as captured var) -> List<JobDetails> via thenApply
3. List<JobDetails> + CompletableFuture<JobsResult> (as captured var) -> JobInfo via thenApply

You can simply unwrap the futures via get() inside those mapper functions because the futures are guaranteed to have completed at that point due to the dependencies of their ancestor futures at that point.

Other approaches using thenCombine and stream reduction would be possible, but more verbose and create more intermediate futures.