Weird WebClient behavior: 1 computer hangs, others don't for same file

Question

I've got a simple C# system tray app which uses System.Net.WebClient to download several files, each in the range of 3-15MB. The WebClient is downloading via HTTPS, and I control the servers from which the downloads are taking place. The files download just fine from a web browser via direct path, and the WebClient is perfectly able to process .DownloadString() methods (i.e., download short text files to string) from this server.

Weirdly, three things have happened in 24 hours, and I'm suspecting some kind of machine-specific config or caching, but I wonder what others might know about WebClient's innards which might permit them to hazard a guess:

1. Development machine #1 had no trouble running this code in Debug mode and downloading the files (call them MegabyteFile1 and MegabyteFile2) until Friday. Downloads took 3-5 seconds, maximum.
2. Suddenly, Development machine #1 stopped downloading MegabyteFile1 on Friday afternoon. Behavior is as follows:
   • WebClient.DownloadFile() hangs indefinitely
   • The file at targetFileFullPath is created but has 0 bytes.
   • Main thread blocks indefinitely.
   • Must exit application.
3. Other development machines with identical OS configurations run the code just fine and download successfully.
4. Manually downloading these files using Chrome or Internet Explorer on DevelopmentMachine1 works just fine, with and without the query parameters.
5. DevelopmentMachine1 will download the file if Fiddler is running (even without having the Fiddler HTTPS certificate impostors installed). In other words, if I try to sniff the wire on this machine, everything works fine.

The pertinent section of the download code reads as follows:

using (WebClient client2 = new WebClient())
{
  client2.DownloadFile(String.Format("{0}?{1}", thePath, queryParams), targetFileFullPath);
}

I've tried the usual sysadmin stuff on this box: rebooted it, cleared the Internet Explorer cache and temp directories, etc.

I know I need to change my strategy for dealing with hung downloads, as I'm sure there will be downloads that hang in the real world. But I'm not confident I'll think of all the corner cases if I don't understand what is causing this particular failure mode. Anyone know enough about what's going on in WebClient to hazard a guess as to why this machine is "special"?

Thanks! -James

Answer 1

Encountered the same problem, but found another solution. Quite complex discussion here: http://social.msdn.microsoft.com/Forums/en-US/a00dba00-5432-450b-9904-9d343c11888d/webclient-downloadstringasync-freeze-my-ui?forum=ncl

In short, the problem is web client is searching for proxy servers and hanging the app. The following solution helps:

WebClient webClient = new WebClient();
webClient.Proxy = null;
... Do whatever else ...

Answer 2

After hours of banging my head on the screen, a pleasant session with WireShark and several separate sessions with Fiddler, I found the answer. Sharing it here in case others have the same problem.

It turns out that any use of HttpWebRequest prior would cause this behavior if the WebResponse object was not .Close()d properly. No matter if you create multiple WebClient or HttpWebRequest objects. They will all fail.

The clue came from the following observation: watching the TCP back-and-forth in WireShark, it became clear that nothing was being sent to the server for the second request (the one that was failing). Why did this work with Fiddler running but not when it wasn't? I suspect that Fiddler is "playing nice" and forcibly closing the connections.

Answer 3

I ran into a similar issue, where WebClient.DownloadFile would timeout when certain web requests happened prior. After searching fruitlessly for a web request response that wasn't properly closed (Using this method), I came across the ServicePointManager.DefaultConnectionLimit property. Setting it higher at the beginning of my application resolved the issue for me, like so:

ServicePointManager.DefaultConnectionLimit = 20