Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache async HttpClient not fast

I am pretty new to Apache http client and am trying to get status code from a website. Found the following example on Apache http tutorial.

import java.util.concurrent.CountDownLatch;
import org.apache.http.HttpResponse;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.concurrent.FutureCallback;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
public class Abc {
    static long d2;
    public static void main(final String[] args) throws Exception {
        d2=System.currentTimeMillis();
        RequestConfig requestConfig = RequestConfig.custom()
            .setSocketTimeout(3000)
            .setConnectTimeout(3000).build();
        CloseableHttpAsyncClient httpclient = HttpAsyncClients.custom()
            .setDefaultRequestConfig(requestConfig)
            .build();
        try {
            httpclient.start();
            final HttpGet[] requests = new HttpGet[] {
                    new HttpGet("http://192.168.26.175:8080/examples/eye/abc10000.jsp")
            };
            final CountDownLatch latch = new CountDownLatch(1);
            for (int v=0;v<1000;v++) {
                httpclient.execute(requests[0], new FutureCallback<HttpResponse>() {

                    public void completed(final HttpResponse response) {
                        latch.countDown();
                        System.out.println(requests[0].getRequestLine() + "->" + response.getStatusLine());
                    }

                    public void failed(final Exception ex) {
                        latch.countDown();
                        System.out.println(requests[0].getRequestLine() + "->" + ex);
                    }

                    public void cancelled() {
                        latch.countDown();
                        System.out.println(requests[0].getRequestLine() + " cancelled");
                    }

                });
            }
            latch.await();
            System.out.println("Shutting down");
        } finally {
            httpclient.close();
        }
        System.out.println("Done");
      long  d1=System.currentTimeMillis();
      System.out.println(d1-d2);
    }

}

Is it really asynchronous or are the calls made serially. What has to be done to make calls asynchronous and faster.

like image 327
Harshit Avatar asked Feb 12 '14 12:02

Harshit


People also ask

Is Apache HttpClient deprecated?

From Apache HTTP Client API version 4.3 on wards, DefaultHttpClient is deprecated.

Is HttpClient asynchronous?

This example demonstrates a basic asynchronous HTTP request / response exchange. Response content is buffered in memory for simplicity. This example demonstrates an asynchronous HTTP request / response exchange with a full content streaming.

What is the default timeout for HttpClient in Java?

The default value is 100,000 milliseconds (100 seconds). To set an infinite timeout, set the property value to InfiniteTimeSpan.

What is CloseableHttpClient in Java?

CloseableHttpClient is an abstract class that represents a base implementation of the HttpClient interface. However, it also implements the Closeable interface.


2 Answers

As the requests are going through the same route following changes will have to be made.

CloseableHttpAsyncClient httpclient = HttpAsyncClients.custom()
            .setDefaultRequestConfig(requestConfig)
            .setMaxConnPerRoute(1000)
            .setMaxConnTotal(1000)
            .build();
like image 188
Harshit Avatar answered Sep 19 '22 02:09

Harshit


First and foremost: CloseableHttpAsyncClient instances are very expensive. Please do NOT create a new CloseableHttpAsyncClient for each and every request. It is like creating a new browser process for each link click, is completely wasteful and is very slow. It is strongly recommended to use the same CloseableHttpAsyncClient instance for the entire lifespan of a logical component.

Pretty much in all cases a blocking client is likely to be considerably faster than a non-blocking (NIO based) one (as long as the number of concurrent requests is below, say, 1000). Unless you are building a proxy of some sort you might be much better served by a blocking HTTP client such as Apache HttpClient.

like image 41
ok2c Avatar answered Sep 18 '22 02:09

ok2c