.NET HttpWebRequest Speed versus Browser

Question

I have a question regarding the performance of the .Net HttpWebRequest client (or WebClient, gives similar results).

If I use HttpWebRequest to request an html page (in this case news.bbc.co.uk) and analyse the speed (using HttpAnalyzer) at which the response is read by the application, this is significantly slower than a browser (Firefox, Chrome, IE) requesting the same resource (all caches cleared etc). The .Net application takes approximately 1.7 seconds versus 0.2 - 0.3 seconds for a browser.

Is this purely down to the speed and efficiency of the code / application or are there any other factors to consider?

Code as follows:

HttpWebRequest request = null;

Uri uriTest = new Uri("http://news.bbc.co.uk");

request = (HttpWebRequest)WebRequest.Create(uriTest);

request.Method = "GET";
request.KeepAlive = true;
request.Headers["Accept-Encoding"] = "gzip, deflate";

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

response.Close();

Answer 1

I'd jack Fiddler in the middle, run the browser request and the .NET request one after the other and make sure you're really getting what you think. It's possible there's redirection or something else hinky going on (maybe browser is pre-appending the '/' while .NET waits for the redir, etc) that isn't immediately visible. I've built huge apps on the .NET HTTP client with nothing like what you describe- something else must be going on.

What happens if you stick '/' on the end of the URL?