Multithread error not caught by catch

Question

Following is a complete console program that reproduces a strange error I have been experiencing. The program reads a file that contains urls of remote files, one per line. It fires up 50 threads to download them all.

static void Main(string[] args)
{
    try
    {
        string filePath = ConfigurationManager.AppSettings["filePath"],
            folder = ConfigurationManager.AppSettings["folder"];
        Directory.CreateDirectory(folder);
        List<string> urls = File.ReadAllLines(filePath).Take(10000).ToList();

        int urlIX = -1;
        Task.WaitAll(Enumerable.Range(0, 50).Select(x => Task.Factory.StartNew(() =>
          {
              while (true)
              {
                  int curUrlIX = Interlocked.Increment(ref urlIX);
                  if (curUrlIX >= urls.Count)
                      break;
                  string url = urls[curUrlIX];
                  try
                  {
                      var req = (HttpWebRequest)WebRequest.Create(url);
                      using (var res = (HttpWebResponse)req.GetResponse())
                      using (var resStream = res.GetResponseStream())
                      using (var fileStream = File.Create(Path.Combine(folder, Guid.NewGuid() + url.Substring(url.LastIndexOf('.')))))
                          resStream.CopyTo(fileStream);
                  }
                  catch (Exception ex)
                  {
                      Console.WriteLine("Error downloading img: " + url + "\n" + ex);
                      continue;
                  }
              }
          })).ToArray());
    }
    catch
    {
        Console.WriteLine("Something bad happened.");
    }
}

On my local computer it works fine. On the server, after downloading a few hundred images, it shows an error of either Attempted to read or write protected memory or Unable to read data from the transport connection: A blocking operation was interrupted by a call to WSACancelBlockingCall..

It seems to be a native error, because neither the inner nor the outer catch catches it. I never see Something bad happened..

I ran it in WinDbg, and it showed the following:

(3200.1790): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
LavasoftTcpService64+0x765f:
00000001`8000765f 807a1900        cmp     byte ptr [rdx+19h],0 ds:baadf00d`0000001a=??
0:006> g
(3200.326c): CLR exception - code e0434352 (first chance)
(3200.326c): CLR exception - code e0434352 (first chance)
(3200.2b9c): Access violation - code c0000005 (!!! second chance !!!)
LavasoftTcpService64!WSPStartup+0x9749:
00000001`8002c8b9 f3a4            rep movs byte ptr [rdi],byte ptr [rsi]

I just turned off Lavasoft, and now WinDbg shows this:

Critical error detected c0000374
(3c4.3494): Break instruction exception - code 80000003 (first chance)
ntdll!RtlReportCriticalFailure+0x4b:
00007fff`4acf1b2f cc              int     3
0:006> g
(3c4.3494): Unknown exception - code c0000374 (first chance)
(3c4.3494): Unknown exception - code c0000374 (!!! second chance !!!)
ntdll!RtlReportCriticalFailure+0x8c:
00007fff`4acf1b70 eb00            jmp     ntdll!RtlReportCriticalFailure+0x8e (00007fff`4acf1b72)
0:006> g
WARNING: Continuing a non-continuable exception
(3c4.3494): C++ EH exception - code e06d7363 (first chance)
HEAP[VIPJobsTest.exe]: HEAP: Free Heap block 0000007AB96CC5D0 modified at 0000007AB96CC748 after it was freed
(3c4.3494): Break instruction exception - code 80000003 (first chance)
ntdll!RtlpBreakPointHeap+0x1d:
00007fff`4acf3991 cc              int     3

Answer 1

Your exception doesn't throw because you, well, do not attempt to get it. WaitAll method is basically a Barrier, which waits (haha) for all tasks to finish. It's void, so you have to save a reference for your tasks for futher actions, like this:

var tasks = Enumerable.Range(0, 50).Select(x => Task.Factory.StartNew(() =>
{
    while (true)
    {
        // ..
        try
        {
            // ..
        }
        catch (Exception ex)
        {
            // ..
        }
    }
})).ToArray();

Task.WaitAl((tasks);

// investigate exceptions here
var faulted = tasks.Where(t => t.IsFaulted);

According MSDN, exceptions are propagated when you use one of the static or instance Task.Wait or Task<TResult>.Wait methods, or .Result property. However, this is not an option for you, as you're using try/catch here. So you need to subscribe to TaskScheduler.UnobservedTaskException event:

TaskScheduler.UnobservedTaskException += TaskScheduler_UnobservedTaskException;

static void TaskScheduler_UnobservedTaskException(object sender, UnobservedTaskExceptionEventArgs e)
{
    Console.WriteLine("Error." + e);
    e.SetObserved();
}

Why does it run without throwing?

This application domain-wide event provides a mechanism to prevent exception escalation policy (which, by default, terminates the process) from triggering.

To make it easier for developers to write asynchronous code based on tasks, the .NET Framework 4.5 changes the default exception behavior for unobserved exceptions. Although unobserved exceptions still raise the UnobservedTaskException exception, the process does not terminate by default. Instead, the exception is handled by the runtime after the event is raised, regardless of whether an event handler observes the exception. This behavior can be configured. Starting with the .NET Framework 4.5, you can use the configuration element to revert to the behavior of the .NET Framework 4 and terminate the process:

<configuration> 
 <runtime> 
  <ThrowUnobservedTaskExceptions enabled="true"/> 
 </runtime> 
</configuration>

Now, back to your code. Consider using a static HttpClient instance instead of HttpWebRequest, as you simply need a result string. This class was designed to be used in multithreaded code, so it's methods are thread-safe.

Also, you should provide a TaskCreationOptions.LongRunning flag to your StartNew method (which is dangerous, by the way, but you still need it):

Specifies that a task will be a long-running, coarse-grained operation involving fewer, larger components than fine-grained systems. It provides a hint to the TaskScheduler that oversubscription may be warranted.

Oversubscription lets you create more threads than the available number of hardware threads. It also provides a hint to the task scheduler that an additional thread might be required for the task so that it does not block the forward progress of other threads or work items on the local thread-pool queue.