Httpclient 4.3 blocking on connection pool

Question

when I use httpclient 4.3 as below

static {
    try {
        SSLContextBuilder builder = new SSLContextBuilder();
        builder.loadTrustMaterial(null, new TrustSelfSignedStrategy());
        SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(builder.build());

        CookieSpecProvider easySpecProvider = new CookieSpecProvider() {

            public CookieSpec create(HttpContext context) {
                return new BrowserCompatSpec() {
                    @Override
                    public void validate(Cookie cookie, CookieOrigin origin) throws MalformedCookieException {
                        // Oh, I am easy
                    }
                };
            }

        };
        Registry<CookieSpecProvider> r = RegistryBuilder.<CookieSpecProvider> create()
                .register(CookieSpecs.BEST_MATCH, new BestMatchSpecFactory())
                .register(CookieSpecs.BROWSER_COMPATIBILITY, new BrowserCompatSpecFactory())
                .register("easy", easySpecProvider).build();
        RequestConfig requestConfig = RequestConfig.custom().setConnectionRequestTimeout(5000)
                .setSocketTimeout(10000).setConnectTimeout(10000).setCookieSpec("easy").setRedirectsEnabled(false)
                .build();

        PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
        cm.setMaxTotal(100);
        cm.setDefaultMaxPerRoute(10);

        client = HttpClients.custom().setConnectionManager(cm).setDefaultCookieSpecRegistry(r)
                .setSSLSocketFactory(sslsf).setDefaultRequestConfig(requestConfig).build();
    } catch (Exception e) {
        logger.error("http client init fail!", e);
    }
}

public static String execute(HttpRequest httpRequest) {
    CloseableHttpResponse response = null;
    HttpGet httpGet = null;
    HttpEntity httpEntity = null;

    try {
        httpGet = new HttpGet(httpRequest.getUrl());

        httpGet.setHeader("Connection", "close"); 
        if (httpRequest.isUseGzip()) {
            httpGet.addHeader("Accept-Encoding", "gzip,deflate,sdch");
        }
        if (!StringUtils.isEmpty(httpRequest.getContentType())) {
            httpRequest.setContentType(httpRequest.getContentType());
        }
        httpGet.addHeader("User-Agent",
                "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63");

        response = client.execute(httpGet);
        httpEntity = response.getEntity();

        byte[] bytes = null;
        try {
            bytes = EntityUtils.toByteArray(httpEntity);
        } catch (Exception e) {
            return null;
        }

        if (response.getStatusLine().getStatusCode() != 200) {
            logger.warn("error! StatusCode: " + response.getStatusLine().getStatusCode() + ", url: "
                    + httpRequest.getUrl());
            return null;
        }

        @SuppressWarnings("deprecation")
        String charset = EntityUtils.getContentCharSet(httpEntity);
        if (StringUtils.isEmpty(charset)) {
            Matcher match = charsetPatterm.matcher(new String(bytes));

            if (match.find()) {
                charset = match.group(1);
            }
        }
        if (!StringUtils.isEmpty(charset)) {
            String strUtf8 = new String(new String(bytes, charset).getBytes(), GlobalConfig.ENCODING);

            return StringEscapeUtils.unescapeHtml4(strUtf8);
        }
    } catch (Exception e) {
        logger.error("error! url [" + httpRequest.getUrl() + "]", e);
    } finally {
        try {
            if (httpEntity != null) {
                EntityUtils.consume(httpEntity);
            }
            if (response != null) {
                response.close();
            }
            if (httpGet != null) {
                httpGet.abort();
            }
        } catch (Exception e) {
            // ignore
        }
    }

    return null;
}

thread will block.. jstack show like this. I just use it to crawl some website. It happens when statusCode is 404.
Using Java Apache PoolingClientConnectionManager leaks the Memory,How to solve it? my problem is similar to this.

"pool-1-thread-10" prio=10 tid=0x00007f7168003000 nid=0x3e4d waiting on condition [0x00007f717c398000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000e69d7350> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:133)
        at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:282)
        at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:177)
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:170)
        at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:102)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.jav
a:244)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:231)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:173)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)

how to solve?

Answer 1

I had encountered similar error. It seems we need to consume HttpEntity before returning. In your case, I see that you are not doing this when you encounter non 200 responses. You were just returning null. You might need to consume this before returning. Also, I would recommend using HttpClientUtils.closeQuietly(response); that is a wrapper around EntityUtils.consume

Answer 2

The condition field that getPoolEntryBlocking is waiting on is a signal that a connection slot is available in the pool. HttpClient doesn't configure a connection timeout by default so await is called on the Future without a timeout, which can cause deadlocks if for any reason connections cannot be returned to the pool. The most likely cause is the lack of timeouts, so configure a sane connection timeout to protect yourself from misbehaving hosts.

If you call CloseableHttpResponse#close you shouldn't see a connection leak. Not consuming the entity will only cause connections to be unable to be reused by the pool, causing more new connections that you otherwise would.

Answer 3

I had the same issue and this was the first response. I used Ironluca's comment to solve my problem but felt this needed a full answer.

The quick start guide has a neat example of how to set up and use a basic HttpClient.

// In order to ensure correct deallocation of system resources
// the user MUST call CloseableHttpResponse#close() from a finally clause.
// Please note that if response content is not fully consumed the underlying
// connection cannot be safely re-used and will be shut down and discarded
// by the connection manager. 
try {
    System.out.println(response1.getStatusLine());
    HttpEntity entity1 = response1.getEntity();
    // do something useful with the response body
    // and ensure it is fully consumed
    EntityUtils.consume(entity1);
} finally {
    response1.close();
}

Looking back at your question it seems you have the close and consume code.
You also have a custom connection pool. It is the same as the default but I guess you have it configured differently.