Strict and reliable timeout on HttpClient

Question

I'm reading a web page using HttpClient like this:

        httpclient = new DefaultHttpClient();
        httpget = new HttpGet("http://google.com");
        HttpResponse response = httpclient.execute(httpget);
        HttpEntity entity = response.getEntity();
        if (entity != null) {
            InputStream PIS = entity.getContent();
        }  

I need a timeout on the entire job (connecting, waiting & reading - All together or separately).
I tried setting timeout parameters just after httpclient = new DefaultHttpClient(); line:

        int timeout=10;
        httpclient.getParams().setParameter("http.socket.timeout", timeout * 1000);
        httpclient.getParams().setParameter("http.connection.timeout", timeout * 1000);
        httpclient.getParams().setParameter("http.connection-manager.timeout", new Long(timeout * 1000));
        httpclient.getParams().setParameter("http.protocol.head-body-timeout", timeout * 1000);

But it didn't worked (It timeouts after about 10 times more than the timeout I set).
So I tried a thread to cancel request after a time using httpget.abort() & httpclient.getConnectionManager().shutdown() just after httpget = new HttpGet("http://google.com"); line like this:

        (new Timer()).schedule(new java.util.TimerTask() {
            public void run() {
                httpget.abort();
                httpclient.getConnectionManager().shutdown();
            }
        },10000);

but it had no effect(Timer runs; but those two lines of code do nothing!)!!
I also tried to use this:

URL url = new URL("http://google.com");
URLConnection con = url.openConnection();
con.setConnectTimeout(10000);
con.setReadTimeout(10000);
InputStream PIS = con.getInputStream();

but it was same as my first try (setting timeout parameters in HttpClient)!!

what is the problem?
How can I solve my timeout problem?

Thanks

Answer 1

Not the solution, but more of an explanation of what is going on.

What you are doing is correct.

First of all, if you are using Log4J, make sure you see everything that HttpClient wants to show you:

log4j.logger.org.apache.http=trace

Then take a look at this class:

http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/conn/DefaultClientConnectionOperator.html

This connection operator is multihome network aware and will attempt to retry failed connects against all known IP addresses sequentially until the connect is successful or all known addresses fail to respond. Please note the same CoreConnectionPNames.CONNECTION_TIMEOUT value will be used for each connection attempt, so in the worst case the total elapsed time before timeout can be CONNECTION_TIMEOUT * n where n is the number of IP addresses of the given host.

That's what most likely is happening in your case.

Also, it is better to use constants from this interface HttpConnectionParams:

SO_TIMEOUT = "http.socket.timeout"
TCP_NODELAY = "http.tcp.nodelay"
SOCKET_BUFFER_SIZE = "http.socket.buffer-size"
SO_LINGER = "http.socket.linger"
SO_REUSEADDR = "http.socket.reuseaddr"
CONNECTION_TIMEOUT = "http.connection.timeout"
STALE_CONNECTION_CHECK = "http.connection.stalecheck"
MAX_LINE_LENGTH = "http.connection.max-line-length"
MAX_HEADER_COUNT = "http.connection.max-header-count"
MIN_CHUNK_LIMIT = "http.connection.min-chunk-limit"

You need only two of them:

HttpConnectionParams.CONNECTION_TIMEOUT
HttpConnectionParams.SO_TIMEOUT

So the best way to solve this is to implement a custom ClientConnectionOperator.resolveHostname method that returns only one IP address.