Max Outgoing Socket Connections in .NET/Windows Server

Question

I have a slightly unusual situation where I'm needing to maintain CLIENT tcp connections to another server for thousands of mobile users on my servers (basically the mobile devices connect to my middle tier server when they are able to, which maintains a more stable connection to the 3rd party server for the mobile devices).

Anyways, I've developed my server application using Async Sockets (wrapped up in a SslStream), and have got 1000 client sessions running full time on it right now. I'm quite happy with the results so far as I'm seeing about 0-10% average cpu usage on a single core processor, and about 60mb of ram being used over time.

My question is, how do I scale this up so I can reach 100,000 or 200,000 or more client sessions being run on my server? Again, this is a bit untraditional, as my server isn't really acting like a server, since I'm worried about outgoing connections, not incoming.

I know that there's a registry setting MaxUserPort that needs to be changed to get beyond the default which seems to be 5000. However, there seems to be another hard limit of 65535, and I'm not too clear on where that limit resides. Is this a limit per network interface? Is it a global Windows limit? Is it a limit per process?

If it is a limit per network interface, can I add multiple network interfaces and bind client session sockets to each interface (eg: 65k on interface 1, 65k on interface 2, etc.)?

I'm also not too sure what, if any socket options or properties I should be setting to help things out. Right now I'm not using any socket options.

I'd really appreciate any thoughts on this subject, as clear advice has been pretty hard to come by on this subject. Thanks!

Answer 1

A Windows machine can easily scale to very high numbers of open connections. The 64k ephemeral port limit is per IP address, not per machine. If you need more ephemeral ports, increase the limits as @SuperTux suggests, but also assign more IPs to the machine. To take advantage, you'll have to manually call Bind() on your client socket and pass a source IP from your pool with free ports (this also implies you'll be responsible for keeping track of available ephemeral port counts per address). Lots of high-end appliance- type devices do this (SNAT pools on load balancers, for instance) to support hundreds of thousands of concurrent connections.

Bookkeeping is a hassle, but better than throwing underutilized hardware at it for every 64k client connections.

Answer 2

65355 is a limit of the IP protocol and more importantly is the limit the TCP/IP stacks of most operating systems impose.

To increase the maximum number of ephemeral ports on Windows, follow these steps:

2. Start Registry Editor.
3. Locate the following subkey in the registry, and then click Parameters: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters

4. On the Edit menu, click New, and then add the following registry entry:

   Value Name: MaxUserPort

   Value Type: DWORD

   Value data: 65534

   Valid Range: 5000-65534 (decimal)

   Default: 0x1388 (5000 decimal)

   Description: This parameter controls the maximum port number that is used when a program requests any available user port from the system. Typically , ephemeral (short-lived) ports are allocated between the values of 1024 and 5000 inclusive.

Normally to scale to more than 65K ports you would use multiple servers in a cluster.