How to execute non blocking HTTP calls in Java?

Question

I have a third party API, which I call using an HTTP GET request. Each request takes a few seconds to get a response.

Currently I am using a CompletableFuture which I am executing on a FixedThreadPool of size 64. This is causing the threads to be blocked until it recieves a response for the GET request, i.e. the threads sit idle after sending the GET response until they recieve a response. So my maximum number of simultaneous requests I can send out is limited by my thread size i.e. 64 here.

What can I use instead of CompletableFuture so that my threads don't sit idle waiting for the response?

Answer 1

As @user207421 says:

• A truly asynchronous (i.e. event driven) HTTP client application is complicated.
• A multi-threaded (but fundamentally synchronous) HTTP client application is simpler, and scales to as many threads as you have memory for.

• Assuming that you have 64 worker threads processing requests, the actual bottleneck is likely to be EITHER your physical network bandwidth, OR your available client-side CPU. If you have hit those limits, then:

  • increasing the number of worker threads is not going to help, and
  • switching to an asynchronous (event driven) model is not going to help.

• A third possibility is that the bottleneck is server-side resource limits or rate limiting. In this scenario, increasing the client-side thread count might help, have no effect, or make the problem worse. It will depend on how the server is implemented, the nature of the requests, etc.

If your bottleneck really is the number of threads, then a simple thing to try is reducing the worker thread stack size so that you can run more of them. The default stack size is typically 1MB, and that is likely to be significantly more than it needs to be. (This will also reduce the ... erm ... memory overhead of idle threads, if that is a genuine issue.)

There are a few Java asynchronous HTTP client libraries around. But I have never used one and cannot recommend one. And like @user207421, I am not convinced that the effort of changing will actually pay off.

---

What can I [do] so that my threads don't sit idle waiting for the response?

Idle threads is actually not the problem. An idle thread is only using memory (and some possible secondary effects which probably don't matter here). Unless you are short of memory, it will make little difference.

Note: if there is something else for your client to do while a thread is waiting for a server response, the OS thread scheduler will switch to a different thread.

So my maximum number of simultaneous requests I can send out is limited by my thread [pool] size i.e. 64 here.

That is true. However, sending more simultaneous requests probably won't help. If the client-side threads are sitting idle, that probably means that the bottleneck is either the network, or something on the server side. If this is the case, adding more threads won't increase throughput. Instead individual requests will take (on average) longer, and throughput will stay the same ... or possibly drop if the server starts dropping requests from its request queue.

Finally, if you are worried of the overheads of a large pool of worker threads sitting idle (waiting for the next task to do), use an execution service or connection pool that can shrink and grow its thread pool to meet changing workloads.