Sending high volumes of POST requests

Question

I am in a situation in which I will be receiving multiple requests, processing the data from the requests, and then forwarding the data to another server using POST. There could possibly be thousands of requests sent to the server at the same time. I have never had to handle this kind of volume before so I had to make somewhat of an (educated) guess on how to proceed.

I am using a Jave EE application server and the data is being forwarded using org.apache.client.HttpClient as POST requests. Each packet of data is relatively small (50-100kb per packet) Here is my current strategy:

When a request comes in, I immediately spawn a new thread in order to process and send the data (one thread per individual request). I am using java.util.concurrent.ThreadPoolExecutor and java.util.concurrent.ArrayBlockingQueue to control the threading and queuing. If all threads in the thread pool are being used, the incoming data is queued. If the queue is full, then the data is dropped (which I'm completely fine with). I am following the documentation on threading using org.apache.client.HttpClient, so all threads share the HttpClient object and only the HttpPost object is being created on each request.

I'm aware that I won't fully know the implications of my implementation until I benchmark this myself, but I wanted to know if there were any red flags before even going that far. To make it clear what my actual questions are:

• Are there any red flags in my approach? (any obvious things I'm doing wrong that may result in a severe performance penalty as I am relatively new to this)

• Is it unwise to give each packet of data its own thread knowing that there will be thousands of them? (the thread count will be capped by the thread pool however)

• Would it be smarter to queue the incoming request and send multiple packets of data per thread instead of one per thread?

Answer 1

Are you running within a web servlet container?

In general, most developers know less about async processing and threading than the teams building todays favourite web servers (tomcat, jetty, even glassfish is pretty good), so for me the only issue with your approach is that you are implementing threading yourself.

If the requests you are reciiving are happening over HTTP then I would look at Servlet 3.0's async capabilities, which allow the HTTP requests to be correctly managed without blocking while you wait for your POST response. There's a good java world article at http://www.javaworld.com/javaworld/jw-02-2009/jw-02-servlet3.html

If your requests are coming in from something other than HTTP (ie JMS or something) then the queue and execute pattern is good, provided you ensure you use thread safe and performant objects (ie ConcurrentHashMap)

For the POST you make, again with HTTPClient there are async executors that are useful - http://hc.apache.org/httpclient-3.x/apidocs/org/apache/commons/httpclient/MultiThreadedHttpConnectionManager.html - describes how to do it.

Have fun!