Virtual thread for reading InputStream from HTTP response

Question

With the java 21 one can convert a blocking IO code to non-blocking one by just executing it in a virtual thread.

Should I simply wrap the HTTP call that returns an InputStream (as in method nonBlockingA), or would it be more efficient to also perform the reading and deserialization of the InputStream in a virtual thread (as in method nonBlockingB)?

In other words, is the reading of InputStream a blocking IO operation?

Keep in mind that the response could be quite large, potentially containing more than 500,000 Strings. I'm also not sure if the used libraries are using any ThreadLocals, which is not recommended with virtual threads

@SuppressWarnings("unchecked")
class Request {
    private final ExecutorService executorService = Executors.newVirtualThreadPerTaskExecutor();
    private CloseableHttpClient httpApacheClient;

    List<String> nonBlockingA() throws Exception {
        InputStream bigInputStream = executorService.submit(this::getResponse).get();
        return deserialize(bigInputStream);
    }

    List<String> nonBlockingB() throws Exception {
        return executorService.submit(() -> {
            InputStream bigInputStream = getResponse();
            return deserialize(bigInputStream);
        }).get();
    }

    private InputStream getResponse() throws IOException {
        return httpApacheClient.execute(new HttpGet("http://random/names/size/500000")).getEntity().getContent();
    }

    private static List<String> deserialize(InputStream body) throws IOException {
        try (InputStreamReader reader = new InputStreamReader(body, UTF_8)) {
            return new Gson().fromJson(reader, List.class);
        }
    }
} 

Answer 1

There is no significant difference between your variants. Any method returning a fully constructed result, like a List<String>, is always a synchronous or blocking method, as there is no way around waiting for any required result of asynchronous or external evaluations, to construct the final result.

Asynchronous methods have to return a Future or similar kind of promise object which allows them to return before the actual result has been computed. This requires the caller to be able to deal with this kind of result. When the caller has to return a fully constructed result to their caller, it brings you back to square one, as it again requires a blocking wait.

So this traditional asynchronous processing requires all processing steps to be implemented as asynchronous operations, chaining callbacks to defer the actual computation to when the inputs are available.

The point of virtual threads is that they do not require you to write such traditional asynchronous methods at all. You can write your operation in a synchronous way. To stay at your example,

List<String> straightForward() throws IOException {
    try(InputStream bigInputStream = getResponse()) {
        return deserialize(bigInputStream);
    }
}

It’s the caller’s responsibility to invoke your code within a virtual thread, to get the benefit. Or, maybe it’s better to say that it’s the caller’s choice to use your method just synchronous or within a virtual thread. But it’s also possible that the caller uses your method in a plain synchronous way but the caller’s caller arranged the call in a virtual thread.

In the best case, 99% of your code does not deal with threads or asynchronous APIs but is written straight-forwardly in a synchronous manner. Only the remaining 1% has to arrange the invocations in virtual threads, like the web server creating a virtual thread per request or such alike.