Does System.Net.Http.HttpClient suffer from HttpWebRequest.AllowWriteStreamBuffering?

Question

I've been trying to use System.Net.Http.HttpClient to POST a larger file (+1GB) but it throws a SystemOutOfMemory exception:

at System.Net.ScatterGatherBuffers.AllocateMemoryChunk(Int32 newSize)
at System.Net.ScatterGatherBuffers..ctor(Int64 totalSize)
at System.Net.ConnectStream.EnableWriteBuffering()
at System.Net.HttpWebRequest.SetRequestSubmitDone(ConnectStream submitStream)
at System.Net.Connection.CompleteStartRequest(Boolean onSubmitThread, HttpWebRequest request, TriState needReConnect)
at System.Net.Connection.SubmitRequest(HttpWebRequest request, Boolean forcedsubmit)
at System.Net.ServicePoint.SubmitRequest(HttpWebRequest request, String connName)
at System.Net.HttpWebRequest.SubmitRequest(ServicePoint servicePoint)
at System.Net.HttpWebRequest.BeginGetRequestStream(AsyncCallback callback, Object state)
at System.Net.Http.HttpClientHandler.StartGettingRequestStream(RequestState state)
at System.Net.Http.HttpClientHandler.PrepareAndStartContentUpload(RequestState state)

Apparently, a similar problem occurs for HttpWebRequest as discussed here: http://support.microsoft.com/kb/908573.

Is there any way to set AllowWriteStreamBuffering of the underlying web request to false? I can't find any.

Cheers,

Answer 1

Just to save time of others interested, I'm answering my own question.

After a few tests the exception seems to be down to the same issue with HttpWebRequest as discussed in the question. I use Microsoft.AspNet.WebApi version 4.0.20710.0.

Below are two equivalent pieces of code; the former fails on large files, whereas the latter works fine.

BTW, despite the issue overall benefits of the HttpClient become really apparent :-)

---

using HttpClient

var clientRef = new System.Net.Http.HttpClient(
    new HttpClientHandler()
    {
        Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword)
    });
clientRef.BaseAddress = new Uri(serverAddress);
clientRef.DefaultRequestHeaders.ExpectContinue = false;
clientRef.PostAsync(
    MyFavoriteURL,
    new System.Net.Http.StreamContent(inputStream)).ContinueWith(
        requestTask =>
        {
            HttpResponseMessage response = requestTask.Result;
            response.EnsureSuccessStatusCode();
        }, TaskContinuationOptions.LongRunning).Wait();

---

using HttpWebRequest

// Preauthenticate
var req  = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(MyFavoriteURL);
req.Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword);
req.Method = "POST";
req.PreAuthenticate = true;
req.Timeout = 10000;
using (var resp = (System.Net.HttpWebResponse)req.GetResponse())
{
     if (resp.StatusCode != System.Net.HttpStatusCode.Accepted && resp.StatusCode != System.Net.HttpStatusCode.OK)
     {
         throw new Exception("Authentication error");
     }
}

// Upload
req = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(MyFavoriteURL);
req.Credentials = new System.Net.NetworkCredential(MyUsername, MyPassword);
req.Method = "POST";
req.PreAuthenticate = true;
req.Timeout = 1200000;
req.ContentLength = inputStream.Length;
req.ContentType = "application/binary";
req.AllowWriteStreamBuffering = false;
req.Headers.ExpectContinue = false;
using (var reqStream = req.GetRequestStream())
{
    inputStream.CopyTo(reqStream);
}

using (var resp = (System.Net.HttpWebResponse)req.GetResponse())
{
    if (resp.StatusCode != System.Net.HttpStatusCode.Accepted && resp.StatusCode != System.Net.HttpStatusCode.OK)
    {
        throw new Exception("Error uploading document");
    }
}