Jersey Client non-blocking

Question

Spawning lots of threads is never a good idea (and when you create too many, you may run out of memory anyway).

Usually, Jersey needs to create one thread per request. And this seems to be the case, whether I use async() ^{(where Jersey creates the threads for me - I've investigated this in a debugger)}, or not ^{(where I obviously have to create the threads myself)}.

So here's one concrete situation where this is not good enough:

I'm HTTP posting to remote servers, at a rate of up to 500 requests/second. But as the response may take some time to arrive (I calculate up to 30 seconds), the total number of threads can reach several thousands easily (at which point, a JVM process usually crashes). Moreover, it's just insane to create so many threads. It should actually be a piece of cake for the available processor/network/OS-resources to deal with that load.

So what I'd like to do, is to just fire off the requests - and be informed by the OS, when the HTTP response arrives.

• As said above, simply using target.request(...).async()....doesn't do the trick (because then, Jersey just spawns its own threads).
• Also, limiting the number of threads via new ClientConfig().property(ClientProperties.ASYNC_THREADPOOL_SIZE, 10) is not helpful at all, because it means, that at most 10 requests will be sent at a time, which is clearly not what I want (it would just pile-up the queue).

I experimented with new ClientConfig().connectorProvider(new GrizzlyConnectorProvider())to get NIO support - but didn't see any difference in behaviour at all.

So is there any way to fire off a request without having to create one extra thread per request?

Answer 1

I am using the CloseableHttpAsyncClient to make async requests to an external service. it works fine with few hundred requests per second, i haven't observed that number of threads like in your case. It's an external dependency which you can integrate through Maven via

<dependency>
  <groupId>org.apache.httpcomponents</groupId>
  <artifactId>httpasyncclient</artifactId>
  <version>4.0.1</version>
</dependency>

Hope this helps.

Answer 2

Make your requests using https://github.com/AsyncHttpClient/async-http-client, it uses netty. It'll fire it's calls off the request thread, and make a call back to your code, so it does not tie up the container request threads.