C# - When to use standard threads, ThreadPool, and TPL in a high-activity server

Question

I've been reading a lot about threading lately as I am looking to develop a high-performance, scalable TCP server capable of handling up to 10,000-20,000 clients, each client of which is consistently communicating bidirectionally to the server with a command-based system. The server will receive a command, and execute either a single (or many) tasks as per the command. My question is how to appropriately make use of the .NET threading constructs for a variety of situations, executing tasks that could take between one minute to several hours, depending on the work being performed.

What's confusing me the most is the fact that everywhere I read, I see something like "use a manually created Thread (or custom thread pool) to handle 'long-running' tasks, and use TPL for short-lived tasks, or tasks that require parallel processing." What exactly is a long-running task? Is that 5 seconds, 60 seconds, an hour?

With what time frame should I be using each of these three methods of creating threads:

• Manually created Threads
• The .NET ThreadPool class
• TPL

Another issue I've contemplated is as follows--say my server does in fact have 20,000 clients connected, each of which sends 1 command (which could translate to one or many tasks) per second. Even with powerful hardware, isn't there a chance that I could be pushing too high of a workload into whatever thread pool / work item queue I have, thereby eventually generating an OutOfMemoryException after the queue slowly fills to the maximum?

Any insight would be greatly appreciated.

Answer 1

Actually, for that scenario all of those are secondary; the first thing you should look at is asyc-IO, aka .BeginRead(...) etc; this allows you to minimise the number of threads by waiting on IO completion ports - much more efficient.

Once you have a complete message, at that scale I would throw the message into a custom thread-pool/synchronized-queue. I would have a controlled number of regular threads (not pool threads or IOCP) servicing that queue to process each item.

As it happens I'm doing something similar (lower scale) at the moment; to prevent memory exploding, I have capped the work queue; if it gets full (i.e. the workers can't keep up) then you might block IOCP for a small while, perhaps with a timeout eventually that tells the client "too busy" at the IOCP layer.