Web Request/Upload Failing at Very End

Question

I find that my HTTPWebRequest Upload is failing at the very end of the upload ... as shown in this video @Screenr

My Code is like below

using (var reqStream = req.GetRequestStream())
{
    BinaryWriter reqWriter = new BinaryWriter(reqStream);
    byte[] buffer = new byte[25600]; // 20KB Buffer
    int read = 0, bytesRead = 0;
    while ((read = memStream.Read(buffer, 0, buffer.Length)) > 0) {
        reqWriter.Write(buffer); // at the very last loop, this line causes the error
        bytesRead += read;
        Debug.WriteLine("Percent Done: " + ((double)bytesRead / memStream.Length * 100) + "% " + DateTime.Now);
    }

I don't know if you need more code, I just don't want to spam code here. the exception below

System.Net.WebException was caught
  Message=The request was aborted: The request was canceled.
  Source=System
  StackTrace:
       at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
       at System.Net.ConnectStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
       at System.Net.ConnectStream.Dispose(Boolean disposing)
       at System.IO.Stream.Close()
       at System.IO.Stream.Dispose()
       at QuickImageUpload.ViewModels.ShellViewModel.UploadImage(String filename, String contentType, Byte[] image) in D:\Projects\QuickImageUpload\QuickImageUpload\ViewModels\ShellViewModel.cs:line 190
  InnerException: System.IO.IOException
       Message=Cannot close stream until all bytes are written.
       Source=System
       StackTrace:
            at System.Net.ConnectStream.CloseInternal(Boolean internalCall, Boolean aborting)
       InnerException: 

Notice the inner exception "Cannot close stream until all bytes are written.". But I havent close any streams in this loop have I?

Answer 1

Well, you're closing the stream by putting it in a using statement - but you should be closing it, so that's unlikely to be the issue.

A couple of points to start with:

• I wouldn't use a BinaryWriter here. It's not buying you anything. Write directly to the stream.

• If memStream is a MemoryStream, you can use WriteTo:

  using (var reqStream = req.GetRequestStream())
  {
      memStream.WriteTo(reqStream);
  }
  

  That means you won't get your diagnostics, but it does make the code simpler :)

Now, as for what's going on... my guess is that you're getting an exception in the call to Write, but that exception is being effectively replaced by an exception being thrown by closing the stream.

I have an idea why an exception may be thrown, too...

Are you setting the content length anywhere? Because you're probably going over it. Look at this line:

reqWriter.Write(buffer);

This is writing the whole buffer out on every iteration of the loop, ignoring how much data you've just read. You've already assigned the number of bytes read into the variable read, but then you're never using that value. You can fix this by changing it to

reqWriter.Write(buffer, 0, read);

... but I personally wouldn't. I'd either just use MemoryStream.WriteTo or write directly to the request stream... as I said before, the BinaryWriter isn't actually buying you anything.

Anyway, in your current code for every request you'd try to write a multiple of 25600 bytes, regardless of your content length. I wouldn't be surprised if the request stream noticed that and threw an exception. Let's pretend we have 30000 bytes of data. I suspect the sequence goes something like this:

• Content length is set to 30000
• Obtain stream and enter using statement
• Read 25600 bytes from memory stream
• Write 25600 bytes out: stream validates that that's correct
• Read 4400 bytes from memory stream (i.e. all the remaining data)
• Try to write 25600 bytes: stream immediately decides that's invalid, and throws an IOException
• The implicit finally block of the using statement then disposes the stream, which closes it
• The stream notices that only 25600 bytes have been successfully written, which isn't enough, so throws an exception

It's this sort of thing that makes it a bad idea for libraries to throw exceptions from Dispose... but assuming I'm right, it does give the amusing problem that you've simultaneously tried to write too much and too little data :)