Why won't my .net app make more than 10 concurrent WebClient requests?

Question

I'm trying to understand the intricacies of async/await in C# and I/O completion ports in Windows, writing code to verify my assumptions along the way.

From what I understand, calling WebClient.DownloadStringTaskAsync(...) will make the current thread register an I/O operation with an I/O completion port (this is probably a little vague, I don't understand the details yet), it will create a Task<string> and it will continue executing code. At some point it will encounter an await for the given task. At that point, it will return from the current method (well, it will exit from some scope, I'm not sure whether this scope could be something other than a method - I should probably inspect the generated state machine to understand that part). Once the I/O operation completes, a thread will be grabbed from the thread pool, it will be passed the result of the I/O operation and it will execute the rest of the scope just mentioned.

I've tried verifying this behaviour, but only got so far as this C# code before something didn't seem right:

class Program
{
    static void Main(string[] args)
    {
        ThreadPool.SetMaxThreads(30, 30);
        Console.WriteLine("Connection limit is {0}", ServicePointManager.DefaultConnectionLimit);
        for (int i = 0; i < 30; i++)
        {
            FetchAsync(i);
        }

        Console.WriteLine("Done starting requests");
        Console.ReadKey();
    }

    private async static void FetchAsync(int num)
    {
        WebClient wc = new WebClient();
        string result = await wc.DownloadStringTaskAsync("http://localhost/slow/index/15");
        Console.WriteLine("Done #{0}", num);
    }
}

As you can see, I am using a WebClient to create 30 requests for a web page (which I know is slow and will take 15 seconds to respond). Running this code, I observe the following behaviour: After 15 seconds the first 10 requests complete. After another 15 seconds, another 10 requests complete and after yet another 15 seconds, the remaining requests complete. Thus, it would seem that there are only 10 outstanding requests at a time (using perfmon I've verified that the web application being called has 10 current requests). Why is that? I was expecting 30 concurrent requests, as I've set the maximum number of threads for the thread pool as per the code above.

This SO question lead me to believe that ServicePointManager might have something to do with it, but since the DefaultConnectionLimit is only 2 I suppose there is no connection.

I realise that this is a pretty long-winded description of a fairly simple question. I hope it will make it easier for someone to point out exactly where my assumptions are wrong.

I'm running this on a Windows 7, 64 bit machine.

Answer 1

You client is able to send arbitrarily many concurrent requests. Your server, though, is localhost. Client Windows OS's have a limitation on the number of concurrent requests in web applications (so that you have to buy a server license). This limit is 10. You can't do anything about it.

Btw, your app is using async IO so you don't need any threads (not even implicit thread-pool threads) while the IO is running. This is not your problem.