Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache HttpClient Multiple polling connections to server strategy

I'm setting up a Java client for a server that I poll periodically and send messages to based on particular response. The strategy I used for the class is as follows:

public class PollingClient {
    private HttpClient client = getHttpClient(); // I get a DefaultHttpClient this way so it's easier to add connection manager and strategy etc to the client later
    private HttpPost httpPost = getHttpPost(); // same idea, I set headers there

    public String poll () {
        List<NameValuePair> formparams = new ArrayList<NameValuePair>();
        formparams.add(new BasicNameValuePair("id", someId));

        String responseString = null;

        try {
            UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(formparams, "UTF-8");        
            httppost.setURI(polluri));
            httppost.setEntity(formEntity);
            HttpResponse response = httpclient.execute(httppost);
            HttpEntity entity = response.getEntity();


            if (entity != null) {
                responseString = EntityUtils.toString(entity); 
            }

            EntityUtils.consume(entity);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public void send(String msg) {
        List<NameValuePair> formparams = new ArrayList<NameValuePair>();
        formparams.add(new BasicNameValuePair("msg", msg));
        formparams.add(new BasicNameValuePair("id", someId));

        String responseString = null;

        try {
            UrlEncodedFormEntity formEntity = new UrlEncodedFormEntity(formparams, "UTF-8");

            httppost.setURI(new URI(URL + "send"));
            httppost.setEntity(formEntity);

            HttpResponse response = httpclient.execute(httppost);
            HttpEntity entity = response.getEntity();


            if (entity != null) {
                responseString = EntityUtils.toString(entity); 
            }

            EntityUtils.consume(entity);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

I start a thread that does the polling at around 3 secs. I can send the messages from the main thread based on the polling results. The code works but keeps giving me the following two exceptions but keeps working in between.

java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.

org.apache.http.NoHttpResponseException: The target server failed to respond.

I could just mute the exceptions but I want to know what's going on. I was unable to find any solutions via Google. I've tried to consume content, make a new HttpPost object in the send method, stuff like that but nothing has helped so far.

What's a good strategy for a situation like this. I currently set the the keep-alive header in the HttpPost object in case that matters. Other than that I don't think there is anything I do. I think this has to do with the strategy overall. I don't want to make new object for every connection but I also don't know what level of reuse is recommended. Thanks for any help. Oh.. and this is HttpClient 4.2.2

like image 227
Saad Farooq Avatar asked Nov 16 '12 21:11

Saad Farooq


People also ask

Is HttpClient multithreaded?

Concurrent execution of HTTP methods HttpClient is fully thread-safe when used with a thread-safe connection manager such as MultiThreadedHttpConnectionManager.

What is keep alive strategy HttpClient?

The Keep-Alive duration determines whether or not the connection is idle, in fact - if the Keep-alive strategy says to keep connections alive for 10 seconds, and we receive responses from the server every 2 seconds, the connection will be kept alive for 10 seconds after the last successful response.

What is HttpClient connection pool?

What is Connection Pooling? The SocketsHttpHandler establishes a pool of connections for each unique endpoint which your application makes an outbound HTTP request to via HttpClient.

Is CloseableHttpClient thread-safe?

[Closeable]HttpClient implementations are expected to be thread safe. It is recommended that the same instance of this class is reused for multiple request executions.


1 Answers

@Saad's answer was very helpful, but I've experienced delays when trying to load test single url with many concurrent requests.

Here is updated version that resolved the problem.

    DefaultHttpClient client = new DefaultHttpClient();
    ClientConnectionManager mgr = client.getConnectionManager();
    HttpParams params = client.getParams();
    int maxConnections = 100;
    params.setParameter(ConnManagerPNames.MAX_CONNECTIONS_PER_ROUTE, new ConnPerRouteBean(maxConnections));
    params.setIntParameter(ConnManagerPNames.MAX_TOTAL_CONNECTIONS, maxConnections);
    ThreadSafeClientConnManager conman = new ThreadSafeClientConnManager(params, mgr.getSchemeRegistry());
    client = new DefaultHttpClient(conman, params);
like image 100
Vadzim Avatar answered Oct 04 '22 18:10

Vadzim