What determines number of simultaneous connections

Question

In a Java servlet environment, what are the factors that are the bottleneck for number of simultaneous users.

1. Number of HTTP connections the server can allow per port
2. Number of HTTP connections the server can allow across several ports (I can have multiple WAS profiles on several HTTP ports)
3. Number of servlets in pool
4. Number of threads configured for WAS to use to service connections
5. RAM available to server (is there any any correletation between number of service threads assuming 0-memory leak in application)

Are there any other factors?

Edited: To leave business logic out of the picture, assume have only one servlet printing one line on Log4j.

• Can my Tomcat server handle 6000 simultaneous HTTP connections? Why not (file handles? CPU time per request?)?
• Can I have thread pool size as 5000 (do idle threads cost CPU/RAM)?
• Can I have oracle connection pool size as 500 connections (do idle connections cost CPU/RAM)?

Is the amount of garbage that is generated for each connection have an impact? For example, if for each HTTP connection 20KB of objects are created and left behind by Tomcat.. then by the time 2500 requests are processed 100MB heap would be used and this may trigger a GC pause of 300ms.

Can we say something like this: if Tomcat uses 0.2 sec of CPU time for processing a single HTTP request, then it would be able to handle roughly 500 http connections in a second. So, 6000 connections would need 5 seconds.

Answer 1

Interesting question, If we leave apart all the performance deciding attributes finally it boils down to how much work you are doing in the servlet or how much time it takes if it has highest I/O, CPU and memory. Now lets move down with you list with the above statement in mind;-

Number of HTTP connections the server can allow per port

There are limit for file descriptors but that again gets triggered by how much time a servlet is taking complete a request or how much time it takes from request first byte receive to finish sending the entire response. Because if it take only 1ms and you are using Netty and persistent connection, you can reach a really high >> 6000.

Number of servlets in pool

Theoretically >> 6000. But how many thread are processing your requests? Is there a thread pool that is burning your requests ? So you want to increase threads, but how much lets say 2000 concurrent threads. Is your CPU behaving poor with context switching ? Is it I/O bound? if yes it makes sense to context switch but then you will be hitting those network limits because a lot of thread waiting on network I/O, so ultimately how much time you spent on a piece of work.

DB

If it oracle, bless you with connection management, you definitely need rigorous monitoring here. Now this is just another limiting factor and can be considered as an just another blocking I/O. By definition of I/O, latency/throughput matters and becomes a bottleneck the moment it becomes the bigger than the smallest piece of work.

So, finally, you need to break down following or more attributes for all the servlets

1. Is it CPU bound? If yes, how much cycles it takes or can it be converted safely to some time unit. e.g. 1ms for just the compute piece of work.
2. Is it I/O bound, If yes similarly find the unit.
3. and others
4. A long list of what you have, e.g. CPU, Memory, GB/s

Now you know how much work needs to be done and all you do is divide by what you have and keep tuning , so that you find out the optimal and also find out what else attribute you have not considered and consider them.