Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache HTTPClient does not make more than 2 connections

I've been trying to implement connection pooling for my application using Apache HTTPClient (v4.1). The problem is that the client always makes only two connections when run, though there are enough threads running parallel. I have been trying to modify the code for a while now, but nothing has helped yet.
I'm using ThreadSafeClientConnManager for connection pooling and set the MaxTotal and DefaulMaxPerRoute to values I want.
Is there anything that comes to your mind first that I might want to check?

Here's that code segment that I use to create the client.

DefaultHttpClient createClient() {
    HttpParams params = new BasicHttpParams();
    HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
    HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);

    params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, new Integer(60000));
    params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, new Integer(60000));
    params.setBooleanParameter(CoreConnectionPNames.TCP_NODELAY, true);

    SchemeRegistry registry = new SchemeRegistry();
    registry.register(new Scheme("https", sf, 6443));
    registry.register(new Scheme("http", 80, PlainSocketFactory.getSocketFactory()));

    ThreadSafeClientConnManager cm = new ThreadSafeClientConnManager(params, registry);
    cm.setMaxTotal(2 * maxConnections);
    cm.setDefaultMaxPerRoute(maxConnections);

    HttpHost localhost = new HttpHost("localhost");
    cm.setMaxForRoute(new HttpRoute(localhost), maxConnections);

    HttpHost sdpTargetHost = new HttpHost("webserviceIP", webservicePort, "https");
    cm.setMaxForRoute(new HttpRoute(sdpTargetHost, null, true), maxConnections);

    return new DefaultHttpClient(cm, params);
}

The client returned by this function is used in Runnables managed by a ThreadPoolExecutor. The Runnables use the client, and has these lines:

HttpResponse response = httpClient.execute(httpPost, context);
HttpEntity entity = response.getEntity();
....
EntityUtils.consume(entity);

From what I know, the EntityUtils.consume(entity) will notify the connection manager that the connection is no longer used, and thus will release the connection to be used by other threads. So I'm guessing the connection management is alright.

I guess I've provided enough info, please tell me if I'm to add anything more.

Thanks

like image 247
shyam Avatar asked Oct 07 '11 06:10

shyam


People also ask

Is HttpClient multithreaded?

HttpClient is fully thread-safe when used with a thread-safe connection manager such as MultiThreadedHttpConnectionManager.

What is the difference between HttpClient and CloseableHttpClient?

CloseableHttpClient is the base class of the httpclient library, the one all implementations use. Other subclasses are for the most part deprecated. The HttpClient is an interface for this class and other classes. You should then use the CloseableHttpClient in your code, and create it using the HttpClientBuilder .

What is HttpClient connection pool?

What is Connection Pooling? The SocketsHttpHandler establishes a pool of connections for each unique endpoint which your application makes an outbound HTTP request to via HttpClient.


1 Answers

OK. I've found the solution, thanks to oleg for pointing out the logging, and to google and all the forums.
All I had to do was define the class with only the connection manager, and then set HttpParams using HttpClient.setParams(). So the code will look something like this:

DefaultHttpClient createClient() {
    SchemeRegistry registry = new SchemeRegistry();
    registry.register(new Scheme("https", sf, 6443));

    ThreadSafeClientConnManager cm = new ThreadSafeClientConnManager(registry);
    cm.setMaxTotal(maxConnections);
    cm.setDefaultMaxPerRoute(maxConnections);

    HttpHost targetHost = new HttpHost("webserviceIP", webservicePort, "https");
    cm.setMaxForRoute(new HttpRoute(targetHost, null, true), maxConnections);

    return new DefaultHttpClient(cm);
}

And right before using the client,

DefaultHttpClient httpClient = createClient();
HttpParams params = new BasicHttpParams();
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
HttpProtocolParams.setContentCharset(params, HTTP.UTF_8);

params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, new Integer(60000));
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, new Integer(60000));
params.setBooleanParameter(CoreConnectionPNames.TCP_NODELAY, true);
httpClient.setParams(params);

There apparently is no difference in the code logically, but this fixed my problem. I presume this probably is some sorta bug in the HttpClient 4.1 API.

like image 164
shyam Avatar answered Oct 20 '22 12:10

shyam