Is there a limit on number of tcp/ip connections between machines on linux?

Question

I have a very simple program written in 5 min that opens a sever socket and loops through the request and prints to the screen the bytes sent to it.

I then tried to benchmark how many connections I can hammer it with to try to find out how many concurrent users I can support with this program.

On another machine (where the network between them is not saturated) I created a simple program that goes into a loop and connects to the server machine and send the bytes "hello world".

When the loop is 1000-3000 the client finishes with all requests sent. When the loop goes beyond 5000 it starts to have time outs after finish the first X number of requests. Why is this? I have made sure to close my socket in the loop.

Can you only create so many connections within a certain period of time?

Is this limit only applicable between the same machines and I need not worry about this in production where 5000+ requests are all coming from different machines?

Answer 1

There is a limit, yes. See ulimit.

In addition, you need to consider the TIME_WAIT state. Once a TCP socket is closed (by default) the port remains occupied in TIME_WAIT status for 2 minutes. This value is tunable. This will also "run you out of sockets" even though they are closed.

Run netstat to see the TIME_WAIT stuff in action.

P.S. The reason for TIME_WAIT is to handle the case of packets arriving after the socket is closed. This can happen because packets are delayed or the other side just doesn't know that the socket has been closed yet. This allows the OS to silently drop those packets without a chance of "infecting" a different, unrelated socket connection.

Answer 2

When looking for the max performance you run into a lot of issue and potential bottlenecks. Running a simple hello world test is not necessarily going to find them all.

Possible limitations include:

• Kernel socket limitations: look in /proc/sys/net for lots of kernel tuning..
• process limits: check out ulimit as others have stated here
• as your application grows in complexity, it may not have enough CPU power to keep up with the number of connections coming in. Use top to see if your CPU is maxed
• number of threads? I'm not experienced with threading, but this may come into play in conjunction with the previous items.

Answer 3

Is your server single-threaded? If so, what polling / multiplexing function are you using?

Using select() does not work beyond the hard-coded maximum file descriptor limit set at compile-time, which is hopeless (normally 256, or a few more).

poll() is better but you will end up with the scalability problem with a large number of FDs repopulating the set each time around the loop.

epoll() should work well up to some other limit which you hit.

10k connections should be easy enough to achieve. Use a recent(ish) 2.6 kernel.

How many client machines did you use? Are you sure you didn't hit a client-side limit?

Answer 4

The quick answer is 2^16 TCP ports, 64K.

The issues with system imposed limits is a configuration issue, already touched upon in previous comments.

The internal implications to TCP is not so clear (to me). Each port requires memory for it's instantiation, goes onto a list and needs network buffers for data in transit.

Given 64K TCP sessions the overhead for instances of the ports might be an issue on a 32-bit kernel, but not a 64-bit kernel (correction here gladly accepted). The lookup process with 64K sessions can slow things a bit and every packet hits the timer queues, which can also be problematic. Storage for in transit data can theoretically swell to the window size times ports (maybe 8 GByte).

The issue with connection speed (mentioned above) is probably what you are seeing. TCP generally takes time to do things. However, it is not required. A TCP connect, transact and disconnect can be done very efficiently (check to see how the TCP sessions are created and closed).

There are systems that pass tens of gigabits per second, so the packet level scaling should be OK.

There are machines with plenty of physical memory, so that looks OK.

The performance of the system, if carefully configured should be OK.

The server side of things should scale in a similar fashion.

I would be concerned about things like memory bandwidth.

Consider an experiment where you login to the local host 10,000 times. Then type a character. The entire stack through user space would be engaged on each character. The active footprint would likely exceed the data cache size. Running through lots of memory can stress the VM system. The cost of context switches could approach a second!

This is discussed in a variety of other threads: https://serverfault.com/questions/69524/im-designing-a-system-to-handle-10000-tcp-connections-per-second-what-problems