What is the difference between the different HttpClients available?

Question

I am trying to write a simple HttpClient program. This is the first time I am working with HttpClient, I am quite confused which jars to include.

I have included the apache-httpcomponents-httpclient.jar and org.apache.commons.httpclient.jar with these ones when I create a HttpClient object I see different methods in the client object

package com.comverse.rht;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.URI;
import org.apache.commons.httpclient.URIException;
import org.apache.commons.httpclient.methods.GetMethod;
import org.apache.commons.httpclient.methods.PostMethod;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpGet;

public class HttpClientTest {

    public static void main(String[] args) throws URIException {
        URI url = new URI("http://www.google.com/search?q=httpClient");
        HttpClient client = new HttpClient();   
        GetMethod get = new GetMethod();
        PostMethod post = new PostMethod();
        String responseString;
        StringBuilder sb = new StringBuilder();
        String line;

        // add request header
        get.setURI(url);
        get.addRequestHeader("User-Agent", "shaiksha429");

        try {
            int respCode = client.executeMethod(get);
            System.out.println("Response Code:" +respCode);
            System.out.println(
                "PCRF HTTP Status" + HttpStatus.getStatusText(respCode)
            );
            responseString = get.getResponseBodyAsString();
            BufferedReader rd = null;
            rd = new BufferedReader(
                new InputStreamReader(get.getResponseBodyAsStream())
            );
            while ((line = rd.readLine()) != null) {
                sb.append(line + '\n');
            }
            System.out.println(sb);
        } catch (HttpException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
}

But when I google I see a different example as below. What is the difference between the two? Why one HttpClient has "execute" and the other has "executeMethod". Which one I need to use?

String url = "http://www.google.com/search?q=httpClient";
HttpClient client = HttpClientBuilder.create().build();
HttpGet request = new HttpGet(url);
// add request header
request.addHeader("User-Agent", USER_AGENT);
HttpResponse response = client.execute(request);
System.out.println("Response Code : " + response.getStatusLine().getStatusCode());
BufferedReader rd = new BufferedReader(
    new InputStreamReader(response.getEntity().getContent())
);
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
    result.append(line);
}

Answer 1

There were a lot of changes from HttpClient version 3 to version 4. The second example is definitely from HttpClient 4, so the first example is probably from the previous version.

Here is code that will do your google search, and read the result into a string

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(60);
connectionManager.setDefaultMaxPerRoute(6);

try (CloseableHttpClient client = HttpClients.custom().setConnectionManager(connectionManager).build()) {

    HttpGet request = new HttpGet("http://www.google.com/search?q=httpClient");
    request.setHeader("User-Agent", "HttpClient");
    try (CloseableHttpResponse response = client.execute(request)) {
        MediaType mediaType = MediaType.parseMediaType(response.getFirstHeader("Content-Type").getValue());
        Charset charSet = mediaType.getCharSet();
        HttpEntity entity = response.getEntity();
        InputStream is = entity.getContent();
        String body = CharStreams.toString(new InputStreamReader(is, charSet));
        System.out.println("body = " + body);
        EntityUtils.consume(entity);
    }
} 

First, you probably want to create a connection pool, so you can reuse the connection if you send multiple requests to the same server. The pool is typically created during application initialisation , for instance as a Spring singleton bean.

Here I used the ClosableHttpClient because it works with resource-try syntax, and you need to close both the httpClient, the response and the inputStream when you are done reading. The HttpClient is actually a lightweight object, the state like socket connection and cookies are stored elsewhere.

I use Spring's MediaType.parseMediaType() to get the char encoding, and Guavas CharStreams to convert the inputStream to a String. In my case google encoded the content using latin-1, because "search" is "søgning" in Danish.

The last step is to use EntityUtils.consume(entity), to ensure that all entity data has been read. If you use connection pooling this is important, because unread data will cause the connection to be thrown away, instead of being reused by the connection manager (this is extremely important if you are using https).

Answer 2

You're using a library whose interface has changed across its major versions. You can't casually copy jars and copy/paste examples without understanding which release you're using and which release an example or snippet was from.

Look at the examples that accompany the latest release and take anything old with a grain of salt.

Apache seems to move especially fast.