Increase number of concurrent connections in tomcat 7

Question

Wondering how to increase the number of concurrent connections in tomcat 7. I did some research and found from apache documentation that setting values to acceptCount, maxConnections and increaseing the maxThreads would do it, but when tried, I was only able to submit 500 requests out of 1000. Below is my snippet from server.xml file -

      <Connector port="443" protocol="HTTP/1.1" SSLEnabled="true"
       maxThreads="2000" acceptCount="1000" maxConnections="1000"
       scheme="https" secure="true"
       keystoreFile="certs/tomcat.keystore" keystorePass="xxxxxxxx"
       clientAuth="false" sslProtocol="TLS" server="Apache" />

Can you please help me finding out the best and easy way to get rid of this one.?

Thanks, Sirish.

Answer 1

Creating a large number of threads¹ and accepting a large number of requests doesn't mean that your server will be able to process the requests

If you have N threads and only M physical processors / cores, then each thread will get 1 processor if M >= N and an average M / N processors if M < N. Assume you have N requests each running on a 1 thread, and each request takes R seconds of CPU time. The average elapsed time T taken to run one request is T = Min(R, R * N / M) seconds. It is fairly clear that as you increase N (the number of active threads and active requests) the average elapsed time T for each individual request increases proportionally.

In addition to that, if you have lots of threads, they will all be using memory, and all will be competing for access to shared data structures ... or the database. An all of this extra resource usage and contention is increasing the overheads of the system as a whole in various ways.

So, what I suspect is happening is that with that number of threads each trying to process requests simultaneously, the time T is starting to approach the client or server side request timeout. (And note that the vagaries of the scheduler, etc mean that the actual time for any given request could be less than or considerably more than the average.) When a request times out, this in turn reduces the throughput in terms of requests that get completed, because the work performed on each timed out requests is (typically) wasted.

---

Unless the requests entail talking to slow external services, I'd advise you to REDUCE the number of threads to no more than 200 ... the Tomcat default². I expect that this will increase the system throughput. It won't necessarily let you process all of those 1000 requests that were launched in that period, but I predict that it will increase the number of requests that are successfully processed.

^{1 - Indeed, increasing the number of threads to 1000 doesn't even mean that you will be able to accept 1000 requests. If you have hundreds of threads in RUNNABLE state, it is likely that Tomcat's listener thread (the one that calls ServerSocket.accept()) will be CPU starved and won't be able to keep up with the request arrival rate.}

^{2 - You will need to do some performance tuning on your system, but I wouldn't be surprised if reducing it even further improved things even more. It will depend on your hardware, your application and (I expect) your backend database.}