HttpWebRequest: The request was aborted: The request was canceled

Question

I've been working on developing a middle man application of sorts, which uploads text to a CMS backend using HTTP post requests for a series of dates (usually 7 at a time). I am using HttpWebRequest to accomplish this. It seems to work fine for the first date, but when it starts the second date I get the System.Net.WebException: The request was aborted: The request was canceled.

I've searched around and found the following big clues:

http://social.msdn.microsoft.com/Forums/en-US/netfxnetcom/thread/0d0afe40-c62a-4089-9d8b-fb4d206434dc

http://www.jaxidian.org/update/2007/05/05/8

http://arnosoftwaredev.blogspot.com/2006/09/net-20-httpwebrequestkeepalive-and.html

And they haven't been too helpful. I've tried overloading the GetWebReuqest but that doesn't make sense because I don't make any use of that function.

Here is my code: http://pastebin.org/115268

I get the error on line 245 after it has run successfully at least once.

I'd appreciate any help I can get as this is the last step in a project I've been working on for sometime. This is my first C#/VS project so I'm open to any tips but I would like to focus on getting this problem solved first.

THanks!

Answer 1

The common solution listed on the Internet appears to be to set the KeepAlive property of the HttpWebRequest to false. This can resolve the problem if the root cause is that the connection is expected to be reused even though it was actually closed automatically after some period of time. However, there is a performance hit to constantly opening and closing connections.

Another possible solution that I used when I encountered this problem was to extend the Timeout properties: WebRequest.ReadWriteTimeout, WebRequest.Timeout, RequestStream.WriteTimeout, and RequestStream.ReadTimeout. Please note these are in milliseconds, so you may want the timeouts to be 1000 * 60 * 10 to represent 10 minutes (or just 600000 if you think you will know what that means...). You can test if this is the more likely cause of the problem by reducing the file size.

BTW. Your code wasn't on the listed website anymore. You may want to include it in the text of your post if it is still an issue.

Answer 2

Here is the only solution works for me:

http://blogs.msdn.com/b/johan/archive/2006/11/15/are-you-getting-outofmemoryexceptions-when-uploading-large-files.aspx

These lines are the key:

    HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(yourUri);
    wr.KeepAlive = false;
    wr.Timeout = System.Threading.Timeout.Infinite;
    wr.ProtocolVersion = HttpVersion.Version10;

And here:

wr.AllowWriteStreamBuffering = false;

Answer 3

A brief summary of my application: I have upto 16 identical threads doing simultaneous HTTP requests. Each of these threads are requesting from different webservers along with unique local endpoints. Now the function making these calls has 3 successive HTTP requests to make (to the same webserver) and do some aggregation.

Based on the solutions posted in the links above, the following combination worked for me.

System.Net.ServicePointManager.DefaultConnectionLimit = 200;
System.Net.ServicePointManager.MaxServicePointIdleTime = 2000;
System.Net.ServicePointManager.MaxServicePoints = 1000;
System.Net.ServicePointManager.SetTcpKeepAlive(false, 0, 0);

HttpWebRequest webRequest1 = (HttpWebRequest)WebRequest.Create("http://" + gatewayIP
 + "/xslt?");
webRequest1.KeepAlive = false;
webRequest1.Timeout = 2000;
//Do other stuff here...
//Close response stream

Thread.Sleep(1000); //This delay seems to help. Obviously very specific to the server

HttpWebRequest webRequest2 = (HttpWebRequest)WebRequest.Create("http://" + gatewayIP
 + "/xslt?");
webRequest2.KeepAlive = false;
webRequest2.Timeout = 2000;
//Do other stuff here...
//and so on...

Answer 4

I am supporting an Azure web service that is accepting files from mobile app and transferring them to Azure blob storage by copying the stream of the incoming request to the stream of an outgoing HttpWebRequest, e.g.

using (Stream requestStream = outgoingRequest.GetRequestStream()) 
{ 
    context.Request.InputStream.copyTo(requestStream)
    requestStream.Flush();
    requestStream.Close();
}

Every once and awhile it was throwing the "The request was aborted: The request was canceled" exception during Close() or in the end of using{} if you don't include Close(). I tried all of the suggested solutions above and none of them worked until I found the actual cause of the error that I would like to share here: Sometimes the InputStream of the incoming request is cut off and does not contain the full file content. When this incomplete stream is copied to the outgoing request's stream the outgoing request throws an error during the Close() or in the end of the using {} block. This is a little bit misleading since the cause is not the outgoing request but
the incomplete stream being copied to it. The way to avoid this is to add a check for the length of the incoming request stream comparing the file length that needs to be stored in the header from the calling app, e.g.

    if (context.Request.InputStream.Length != int.Parse(context.Request.Headers["file-length"])) 
    { 
        //handle this case here, e.g.
        context.Response.StatusCode = 500;
        context.Response.StatusDescription = "(500) Internal Server Error";
        context.Response.End();
    }
    else
    {
         //proceed with copying the stream (the first code snippet)
    }