How to handle file HttpClient.PostAsync file upload under poor network conditions?

Question

I'm working on developing a mobile app that is centered around uploading multiple photos to a web api. I'm using Xamarin.Forms and System.Net.Http.HttpClient, and Clumsy to simulate poor network conditions (lag, dropped packets, out-of-order packets). The app was originally written with Titanium, and worked fine for most users, but some users on poor mobile networks were getting frequent errors. Going forward we are porting to Xamarin and trying to accommodate users with poor connectivity.

using (var httpClient = CreateClient())
        {
            httpClient.Timeout = TimeSpan.FromMinutes(5);
            using (var formData = new MultipartFormDataContent())
            {
                // add required fields via formData.Add() ...

                var httpContent = new ByteArrayContent(imageData);
                formData.Add(httpContent, "file", Guid.NewGuid() + ".jpg");

                try
                {
                    var response = await httpClient.PostAsync("fileupload", formData).ConfigureAwait(false);

                    if (response.IsSuccessStatusCode)
                    {
                        responseObject = await ResponseMessageToResponseModel(response).ConfigureAwait(false);
                    }
                }
                catch (HttpRequestException ex)
                {
                    Debug.WriteLine("HttpRequestException");
                }
                catch (TaskCanceledException ex)
                {
                    Debug.WriteLine("TaskCanceledException");
                }
            }
        }

What I'm finding is that everything works as expected under normal conditions, however; when enabling Clumsy with "lag, drop, out-of-order" and attempting the upload the PostAsync() never completes and eventually times out with TaskCanceledException. The odd thing is that the file ends up on the server.. so the POST data apparently made it through okay.

I'm guessing that packets dropped in the response from the server means the HttpClient never receives a proper response and continues to wait for one until it times out.

To get to the point, I'm wondering if anyone has any ideas on how to make this process as bullet-proof as possible. Just catching the timeout and trying again doesn't work very well if the file made it through the first time. Any thoughts?

Also, any info on how HttpClient handles dropped/out-of-order packets so I can better understand what's happening would be great as well.

Answer 1

One thing with HttpClient I banged my head at a while ago was special(read uncommon) handling of POST requests. When sending POST request it first sends the headers (including ContentLenght and special Expect: 100-continue header) to the server but without the body. And waits for server to respond with status code 100 if request is acceptable. After that it starts sending the body. Additional info here: MSDN Page for ServicePointManager.Expect100Continue MSDN blog post with some details

In my case the problem was that this part of the protocol wasn't handled particulary well by the backend service (Play framework) I was talking to when request size was too large for it to handle. It was not returning any error. And request simply timed out. So disabling it with ServicePointManager.Expect100Continue = false; well ahead of sending any request to this host solved the problem for me. At least now it was returning something.

If that doesn't help than the best thing I can recommend is looking with wireshark or something similar at what is going on on the wire. Provided it plays nice with this Clumsy tool you're using. (Thanks for the link by the way. Was looking for something like this myself)