Should I use Threads or Tasks - Multiple Client Simulation

Question

I am writing a client simulation program in which all simulated client runs some predefined routine against server - which is a web server running in azure with four instances.

All simulated client run the same routine after getting connected to server.

At any time I would like to simulate 300 to 800 clients using my program.

My question is: Should I create N instances of client class and run them in N different threads? OR

Should I use Task Library to do the things?

Answer 1

You certainly should not create 800 threads.

Let's take a step back here. You have a device called a "server" which takes in "requests" from "clients" and sends out "responses" back to those clients. Let's suppose that the requests are pieces of paper delivered by the post office, and the responses are boxes containing books, also delivered by the post office.

You wish to simulate 800 clients in order to test the server.

Let's suppose that a thread is a person and a processor is a chair. A person can only do work while sitting in the chair.

Creating 800 threads is the equivalent of going out and hiring 800 people, and paying each of them to send a letter to the server. But you only have four chairs, so those 800 people have to all take turns using the chairs.

That would be a ludicrous solution in real life. Threads, like people, are insanely expensive. You should be minimizing the number of threads you create.

So, should you instead create 800 tasks via the task factory and let the TPL parallelize them for you?

No, you should not do that either. The TPL has a pool of people (threads) to draw from, and it tries to arrange things so that there are no more people on the payroll than there are chairs for them to sit in. But your task is not "chair bound" -- the people are going to sit in the chair, send the request to the server, and then get out of the chair while they wait for the response to come back. While they are waiting, the TPL now has to hire more people to service the additional tasks.

Hitting a web server is I/O bound; you should only create thread-pooled tasks for tasks that are CPU bound.

The right solution is to hire two people.

One person -- the "I/O completion thread" -- does nothing but drop requests in the mailbox and check for incoming packages. The other person -- the "simulation" person -- works out what the right "schedule" is for simulating 800 clients. The simulation person works out the schedule, and then goes to sleep. She wakes up when it is time to send another request to the server. When she wakes up, she tells the I/O completion thread to drop this letter in the mailbox, and wake her up when the response comes in. She then goes back to sleep until either it is time to send another request, or a response comes in that needs to be verified.

What you should do is either (1) get the beta version of C# 5 and use async/await to create tasks that send requests to the server, and then yield control back to the message loop until either it is time to send another request or a response comes in. Or, if you don't want to use C# 5, you should create a Task Completion Source, and set up tasks that have the right continuations.

In short: the right way to handle many parallel I/O tasks is to create a very small number of threads, each of which does a very small amount of work at a time. Let the I/O completion thread handle the details of the I/O. You do not need to hire 800 people in order to simulate sending 800 letters. Hire two people, one to watch the mailbox and one to write the letters.