Async webrequest times out => Crashes IIS

Question

I've got a web app, that gets data from external services. The request itself happens like the code below - quite straightforward as far as I can see. Create a request, fire it away asynchronously and let the callback handle the response. Works fine on my dev environment.

public static void MakeRequest(Uri uri, Action<Stream> responseCallback)
        {
            WebRequest request = WebRequest.Create(uri);
            request.Proxy = null;
            request.Timeout = 8000;
            try
            {
                Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null)
                    .ContinueWith(task =>
                                      {
                                          WebResponse response = task.Result;
                                          Stream responseStream = response.GetResponseStream();
                                          responseCallback(response.GetResponseStream());
                                          responseStream.Close();
                                          response.Close();
                                      });
            } catch (Exception ex)
            {
                _log.Error("MakeRequest to " + uri + " went wrong.", ex);
            }
        }

However external test environments and the production environment could, for reasons beyond me, not reach the target URL. Fine, I thought - a request timeout won't really hurt anyone. However, it seemed that every time this request timed out, ASP.NET crashed and IIS was restarted. The event log shows me, among other things, this stacktrace:

StackTrace:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.get_Result()
   at MyApp.AsyncWebClient.<>c__DisplayClass2.<MakeRequest>b__0(Task`1 task)
   at System.Threading.Tasks.Task`1.<>c__DisplayClass17.<ContinueWith>b__16(Object obj)
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()

InnerException: System.Net.WebException

Message: Unable to connect to the remote server

StackTrace:    at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endMethod, TaskCompletionSource`1 tcs)

InnerException: System.Net.Sockets.SocketException

Message: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

StackTrace:    at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)

..so it all boils down to a SocketException, it seems to me. And right afterwards (within the same second) another event (which I'm guessing is relevant) is logged:

Exception Info: System.AggregateException
Stack:
   at System.Threading.Tasks.TaskExceptionHolder.Finalize()

This is sort of beyond me as I'm no master of async code and threading, but that a timeout from a web requests causes IIS to crash seems very weird. I reimplemented MakeRequest to perform the requests synchronously, which works out great. The request still times out in those environments, but no damage is done and the app continues to run happily forever after.

So I've sortof solved my problem, but why does this happen? Can anyone enlighten me? :-)

Answer 1

Your continuation needs to handle the fact that .Result might reflect an exception. Otherwise you have an unhandled exception. Unhandled exceptions kill processes.

.ContinueWith(task =>
{
    try {
        WebResponse response = task.Result;
        Stream responseStream = response.GetResponseStream();
        responseCallback(responseStream);
        responseStream.Close();
        response.Close();
    } catch(Exception ex) {
        // TODO: log ex, etc
    }
});

your old exception handler only covers the creation of the task - not the callback.