Handling 7000+ clients? -MultiThreading (TCP, high-traffic)

Question

I'm going to create a network system that can handle 7000+ tcp socket client with 5KB/s input (clients sends). I've looked into this question: Link>>. They said "create 1024 thread to handle 1024 clients". I know there is a method named "select()" and I think I can't open 7000+ threads to handle 7000+ clients because my processor (or server) have got only 8CPU and that means 7000+ threads are a big mistake. Now I think I'll create ~1000 thread and I'll handle every 7 socket group in this threads. But now here is the question: If I have same application but I've 2CPU processor, I can't get maximum performance with 1000 threads, I should create (maybe) 500 threads. Otherwise if I've 8CPU processor, I can't get max. performance with 1000 threads and I need to create (maybe) 2000 threads to handle sockets. How can I understand "this processor can handle X threads"? And is this way true?

EDIT: I think I can create a profiller that is watching program. Namely, every thread logging "I have finished my job in X seconds." and profiller handling this messages and decides to create a thread or killing a thread. But how to understand threads' (or CPU) status?

Answer 1

There is no way a single server can handle 35 Gb/sec traffic (and even if there is, it will be extremely expensive).

The way I'd approach this problem is:

• Understand API
• Understand protocols, select the best one for the job
• Understand compression, security aspects (encryption, authentication), licensing
• Figure out how you going to load balance your servers
• Write a prototype server and client
• Generate some load on the server and understand limitations of a single instance
• Hide N servers behind load balancers and observe their behaviour

Things you want to concentrate from the first line of code:

• How you going to scale this horizontally
• What are your performance metrics

Edit

So it's in KB, which is much better :). I would still recommend thinking about LB upfront.

There are different technologies and protocols to help you write efficient application. From personal experience I would prefer HTTP to TCP. There are many good load balancers available, adding compression, encryption, authentication is matter of days.

I also heard node.js is superfast if you do any IO operations for processing client requests.

The servers that I wrote were ASP.NET Web API applications, each processing a few MB/sec. It was trivial to hide servers behind load balancers (I used HAProxy and nginx but surely there are many others available. I also worked with C++ socket server and the development time for it was considerably longer. My point is, if you can use a high-level language, prefer it to lower-level language. This will shorten you dev time (in my case by a factor of 10x!) and make life easier.