Apache HTTPClient 4.3.3 execute method for a GET request blocks and never returns

Question

Facing this weird issue since morning, I am making a REST call to a particular endpoint which gives responses in pages so i need to make calls again and again until all pages are completed. My code works fine and dandy until the last page after the last page the next request I make (which should return a blank page) by httpClient.execute(httpGet); is blocked permanently and never comes back or neither throws any exception. If I set connection request time out then the last call is not blocked and fails saying timeout but I am not getting why that last call is not working If I make the same last call from firefox RESTClient it works. Kindly help.

I also tried adding different log levels like System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.SimpleLog"); and other logs to debug but no luck.

Below is my code.

private static Servers fetchServers(String token,String endpoint) throws Exception{
    JsonReader jreader = null;
    InputStreamReader isr = null;
    CloseableHttpClient httpClient = null;
    try{
        URI uri = new URI(endpoint);

        /** accepting all certificates */
        httpClient = getSecuredHttpClient();

        uri = new URIBuilder(uri)
                .setParameter("limit", "1")//Page limit, for testing have kept just 1 since i have only 2 records.
                .build();

        HttpGet httpGet = new HttpGet(uri);

        httpGet.setHeader(HTTP.CONTENT_TYPE, "application/json");
        httpGet.setHeader("X-Auth-Token", token);
        httpGet.setHeader("accept", "application/json");
        httpGet.setHeader(HTTP.USER_AGENT, "python-neutronclient");

        Servers cloudServers = new Servers();
        cloudServers.setServers(new ArrayList<Server>());

        boolean nextPage = false;

        do {
                HttpResponse resp = httpClient.execute(httpGet);//this is where it gets blocked for last page.
                if(resp.getStatusLine().getStatusCode() == HttpStatus.SC_OK){
                    isr = new InputStreamReader(resp.getEntity().getContent());
                    jreader = new JsonReader(isr);
                    Gson gson = new GsonBuilder().registerTypeAdapter(Address.class, new AddressAdapter()).create();

                    Servers servers = gson.fromJson(jreader, Servers.class);
                    cloudServers.getServers().addAll(servers.getServers());

                    if(servers.getServersLinks() == null || servers.getServersLinks().size()==0)
                        nextPage = false;
                    else if(servers.getServersLinks().get(0).getRel().equals("next"))
                        nextPage = true;

                    uri = new URI(servers.getServersLinks().get(0).getHref());//this gives the url for next page.
                    httpGet.setURI(uri);
                }else{
                    break;
                }
        } while (nextPage);
        return cloudServers;
    }catch(Exception e){
        throw e;
    }finally{
        if(null != jreader){
            jreader.close();
        }
        if(null != isr){
            isr.close();
        }
        if(null != httpClient){
            httpClient.close();
        }
    }
}

Answer 1

I am fairly sure that your code simply leaks and eventually runs out of connections.

do {
    CloseableHttpResponse resp = httpClient.execute(httpGet);
    try {

    // Do what you have to do 
    // but make sure the response gets closed no matter what
    // even if do not care about its content

    } finally {
        resp.close();
    }        
} while (nextPage);