c# Chunked transfer of file upload in combination with access to route parameters in ServiceStack

Question

I'm looking to use the IRequiresRequestStream interface to enable large file uploads (video files) using ServiceStack (v3) and chunked transfer encoding. The standard file upload can't seem to cope with some of the larger video files our customers are uploading, so we are looking to enable chunked transfer encoding for these files.

I have successfully tested the chunked transfer encoded file upload, but there are a number of parameters that also need to be sent across with the file.

Since IRequiresRequestStream bypasses the ServiceStack request object parser, any other parameters in the request object alongside the Stream are obviously not populated. As a work around I can see the following options:

1. Query String parameters, accessible via this.Request.QueryString collection
2. Custom header parameters, accessible via this.Request.Headers collection
3. Path, accessible via RequestBinder??

I've already managed to implement options 1 and 2, but somehow neither feel quite RESTful enough. I'd prefer to use the Path -> RequestDTO, but I'm struggling with the RequestBinder.

Service:

public object Any(AttachmentStreamRequest request)
{
    byte[] fileBytes = null;

    using (var stream = new MemoryStream())
    {
        request.RequestStream.WriteTo(stream);
        length = stream.Length;
        fileBytes = stream.ToArray();
    }

    string filePath = @"D:\temp\test.dat";
    File.WriteAllBytes(filePath, fileBytes);

    var hash = CalculateMd5(filePath);
    var requestHash = this.Request.QueryString["Hash"];
    var customerId = this.Request.QueryString["CustomerId"];
    var fileName = this.Request.QueryString["FileName"];

    // nicer would be
    // var requestHash = request.Hash;
    // var customerId = request.CustomerId;

    // save file....

    // return response
    return requestHash == hash
               ? new HttpResult("File Valid", HttpStatusCode.OK)
               : new HttpResult("Invalid Hash", HttpStatusCode.NotAcceptable);
}

Request:

[Route("/upload/{CustomerId}/{Hash}", "POST", Summary = @"POST Upload attachments for a customer", Notes = "Upload customer attachments")]
public class AttachmentStreamRequest : IRequiresRequestStream
{
    // body
    public Stream RequestStream { get; set; }

    // path    
    public int CustomerId { get; set; }

    // query
    public string FileName { get; set; }

    // query
    public string Comment { get; set; }

    // query
    public Guid? ExternalId { get; set; }

    // path
    public string Hash { get; set; }
}

WebClient:

private static async Task<string> SendUsingWebClient(byte[] file, string hash, customerId)
{
    var client = (HttpWebRequest)WebRequest.Create(string.Format("http://localhost.fiddler:58224/upload/{0}/{1}", customerId, hash));
    client.Method = WebRequestMethods.Http.Post;
    client.Headers.Add("Cookie", "ss-pid=XXXXXXXXXXX; ss-id=YYYYYYYYYY");

    // the following 4 rows enable streaming 
    client.AllowWriteStreamBuffering = false;
    client.SendChunked = true;
    client.ContentType = "application/json";
    client.Timeout = int.MaxValue;

    using (var fileStream = new MemoryStream(file))
    {
        fileStream.Copy(client.GetRequestStream());
    }

    return new StreamReader(client.GetResponse().GetResponseStream()).ReadToEnd();
}

I'm guessing the simple direction to take is something along the following lines, but it seems like a kludge.

RequestBinders.Add(typeof(AttachmentStreamRequest), httpReq => { 
    var dto = new AttachmentStreamRequest(); 
    var segments = base.Request.PathInfo.Split(new[] { '/' }, StringSplitOptions.RemoveEmptyEntries);

    dto.CustomerId = segments[1].As<int32>();
    dto.Hash = segments[2].As<string>();

    // Stream copy to dto.RequestStream and other params etc....

    return dto;
});

I've done a bit of Googling for examples of RequestBinders in this scenario. I'm sure there must be inbuilt ServiceStack methods for parsing the Path, but I'm struggling with it. Does anyone have an example they would like to share?

Answer 1

Recently I also investigated using Chunked transfer with custom headers. Unfortunately, I found out that it's not supported out-of-the-box in HttpWebRequest class nor in .NET Framework in general. The only solution that worked for me was to implement Chunked Transfer HTTP communication over TCP. It's not as complex as it sounds in the begginning. You just need to open TCP client connection, format the headers as needed, split your stream by chunks and send it.

Here is the definition of Chunked Transfer protocol:

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding