Apache HttpClient loadbalancing pooled connections

Question

We are using persistent connections, and have tried forcing connections to get dropped after x amount of time. While I see we can in theory use ConnectionKeepAliveStrategy, form what I can tell this only applies after a response..i.e. while the connection is idle.

The issue we are having..

Assume 1 client, hitting 2 servers (A,B) via a loadbalancer. When one of the servers goes offline (B), all the new connections are established on the against the server (A). Now when the the other server (B) comes back online, it will remain idle as all the connections are on the other server (A). As long the client keeps accessing a connection below the idle timeout/keepalive, this will continue, leaving the B server idle (aka with zero connections).

What we want to do..is force the all persistent connections to periodically get closed out (within a 'randomized time window'. Ideally we do not want all connections to reset at the same time). Any suggestions on doing this?

We tried extending HttpClientConnectionManager, and tracking how long a connection had been open for, then close it out after x amount time...however this does not seem to work. I'm guessing this is because HttpClientConnection is not in fact the actual connection, but is rather a proxy and looks like underneath this proxy, it is actually 'using' one of the established connections, as such making its impossible to actually track the time those underlying connections have been established for.

Thoughts?

Right now I'm toying with idea of simply calling: HttpRequestBase.abort() on 1 connection per minute once we have executed a request on it, which I think would get us some what closer to desired behavior.

Answer 1

I don't think you're using the load balancer properly. If you're wired like this:

              +--> SA
C <---> LB <--+
              +--> SB

The client can have persistent connections to the load balancer. The LB<-->SA and LB<-->SB connections can be with persistent connections or not, doesn't matter. The Load balancer should understand HTTP and route on that layer, and not just TCP connections. Thus two incoming (to the LB) HTTP requests on the same persistent connection can be routed to two separate servers.

Answer 2

One can limit the total life time of a connection by using TTL (time to live) parameter.

HttpClientBuilder.create()
        .setConnectionTimeToLive(1, TimeUnit.MINUTES)
        .build();

That will force all connections to get renewed after one minute.