Why does async-http-client does not throttle my requests?

Question

I have an Akka actor that owns an AsyncHttpClient. This actor must handles a lot of asynchronous requests. Because my system cannot handle thousands of requests simultaneously, I need to limit the number of concurrent requests.

Right now, I'm doing this :

AsyncHttpClientConfig config = new AsyncHttpClientConfig.Builder().setAllowPoolingConnection(true)
                                                                  .addRequestFilter(new ThrottleRequestFilter(32))
                                                                  .setMaximumConnectionsPerHost(16)
                                                                  .setMaxRequestRetry(5)
                                                                  .build();
final AsyncHttpClient httpClient = new AsyncHttpClient(new NettyAsyncHttpProvider(config));

When my actor receives a message, I use the client like this :

Future<Integer> f = httpClient.prepareGet(url).execute(
   new AsyncCompletionHandler<Integer>() {
      @Override
      public Integer onCompleted(Response response) throws Exception {
         // handle successful request
      }

      @Override
      public void onThrowable(Throwable t){
         // handle failed request
      }
   }
);

The problem is that requests are never put in the client queue and are all processed like the configuration doesn't matter. Why doesn't this work as it should?

Answer 1

From the maintainer:

setMaxConnectionsPerHost only caps the number of connections that can be open to a given host. There's no built-in queuing mechanism for requests that might need a connection while there's none available.

So basically, it's a hard limit. Also, in versions of the library prior to, I believe, 1.9.10, the maximumConnectionsPerHost field was not being properly utilized by the code to limit the number of concurrent connections per host. Instead, there was a bug where the client only looked at the maximumConnectionsTotal field.

Link to issue referenced on GitHub