ASP.Net Web API doesn't read all bytes from StreamContent

Question

I have an ASP.Net Web API set up on my website that is used to communicated with a WPF desktop application. I have an action setup on the API to receive binary files from the client application. However in some (seemingly random) cases when I get all the bytes from the request not all the bytes are read. Hopefully you can give me an idea of how to do this in a way that will work all of the time. Here's the code:

Client Side:

public static SubmitTurnResult SubmitTurn(int turnId, Stream fileStream)
{
    HttpClient client = CreateHttpClient();

    HttpContent content = new StreamContent(fileStream);
    content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
    content.Headers.ContentDisposition.FileName = "new-turn.Civ5Save";
    content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
    content.Headers.ContentLength = fileStream.Length;

    HttpResponseMessage response = client.PostAsync(
        string.Format("SubmitTurn?authKey={0}&turnId={1}",
                        LocalSettings.Instance.AuthenticationKey,
                        turnId
                        ),
        content
    ).Result;

    response.EnsureSuccessStatusCode();

    return response.Content.ReadAsAsync<SubmitTurnResult>().Result;
}

SubmitTurnResult is an enum that defines the result on the server, turnId is the ID for the entity this file is attached to, and fileStream is an actual FileStream reading the bytes of disk.

Server Side:

[HttpGet, HttpPost]
public SubmitTurnResult SubmitTurn(string authKey, int turnId)
{

    try
    {
        bool worked = false;
        int gameId = 0;

        using (GmrEntities gmrDb = new GmrEntities())
        {
            var player = gmrDb.Users.FirstOrDefault(u => u.AuthKey == authKey);
            if (player != null)
            {
                var turn = player.Turns.FirstOrDefault(t => t.TurnID == turnId);
                if (turn != null)
                {
                    byte[] saveFileBytes = null;

                    using (MemoryStream tempStream = new MemoryStream())
                    {
                        var task = this.Request.Content.CopyToAsync(tempStream);
                        task.Wait();

                        saveFileBytes = tempStream.ToArray();
                        tempStream.Close();
                    }

                    if (saveFileBytes.Length != this.Request.Content.Headers.ContentLength.Value)
                    {
                        throw new Exception(string.Format("Byte array length ({0}) not equal to HTTP content-length header ({1}). This is not good!",
                                    saveFileBytes.Length, this.Request.Content.Headers.ContentLength.Value));
                    }

                    worked = GameManager.SubmitTurn(turn, saveFileBytes, gmrDb);

                    if (worked)
                    {
                        gameId = turn.Game.GameID;

                        gmrDb.SaveChanges();
                    }
                }
            }
        }


        return SubmitTurnResult.OK;
    }
    catch (Exception exc)
    {
        DebugLogger.WriteExceptionWithComments(exc, string.Format("Diplomacy: Sumbitting turn for turnId: {0}", turnId));

        return SubmitTurnResult.UnexpectedError;
    }
}

Answer 1

As noted in my previous comment, we ran into this same behavior with StreamContent, but when streaming a response from a Windows Server 2003 Web API service. It doesn't repro on 2008. Actually, it also repros on Windows Server 2008 if I configure the VM with a small amount of RAM (712 MB), but with 4 GB of RAM it doesn't repro. Also, we found that this only repros with a FileStream. Converting the FileStream to a MemoryStream bypasses the issue (at the expense of memory of course). We found that when the response stream terminates early, it's always on a 4096-byte boundary, and it hits a cap at around 3.5MB.

Here's the workaround that fixed things for me, tailored to your code example:

public static SubmitTurnResult SubmitTurn(int turnId, Stream fileStream)
{
    HttpClient client = CreateHttpClient();

    var memoryStream = new MemoryStream((int)fileStream.Length);
    fileStream.CopyTo(memoryStream);
    fileStream.Close();
    memoryStream.Seek(0, SeekOrigin.Begin);
    HttpContent content = new StreamContent(memoryStream);

If desired, you can conditionally do the MemoryStream copy only when Stream is a FileStream.