Correct way to implement HTTP Connection Pooling

Question

I am using Apache HTTP Client for connection pooling during my REST API calls into certain web services.

Strange thing is that in spite of me using HTTP Connection Pooling there are no gain in my performance.

I am using Apache HTTP Client to connect to my web services, and the code is as follows from there documentation :

PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();

cm.setMaxTotal(200);

cm.setDefaultMaxPerRoute(20);

HttpHost host = new HttpHost("abc.com", 80);
cm.setMaxPerRoute(new HttpRoute(host), 50);

CloseableHttpClient httpClient = HttpClients.custom()
        .setConnectionManager(cm)
        .build();

I am using Spring's RestTemplate to wrap around the HttpClient implemenation of Apache using Spring's HttpComponentsClientHttpRequestFactory.

But even if I use no connection pooling ie. use the SimpleClientHttpRequestFactory of Spring, I get no performance advantage.

My connections still take the same amount of time to complete.

Is what I have done the correct way to implement HTTP Connection Pooling? Am I doing something wrong?

Please let me know if any further info is required from my side.

Answer 1

Beware of how HTTP Client pools work, it may be improving performance during a short period of time. Check the analysis below:

From PoolingHttpClientConnectionManager javadocs

The handling of stale connections was changed in version 4.4. Previously, the code would check every connection by default before re-using it. The code now only checks the connection if the elapsed time since the last use of the connection exceeds the timeout that has been set. The default timeout is set to 2000ms

From the pool performance perspective it means that a connection to a particular route will be reused as long as the manager considers that route as "active" during a period of 2 seconds by default.
After 2 seconds of inactivity connections to that route will be considered stale and discarded thus incurring in a connection init penalty next time that route is requested.

In other words, out of the box, the pool improves performance for connections after the first during 2 seconds. Heavy duty routes are the most benefited.

As a simple test, set your pool size to an small value, like 5 max. Send 5 requests and check the number of established connections to that route, on linux

watch "ss -ant dst <your route IP>"

watch "netstat -ant | grep <your route IP>"

You should see 5 connections. Wait 10 or 20 seconds and send 2 requests to the same route, you should see those 5 connections closed and 2 new created. It's also possible to observe that with debug logging. Here is a good article as reference.
Apache official docs on http logging.