Send large file from WebAPI.Content Length is 0

Question

I am trying to send large file (GB) from one WebAPI (.NET Core) to another WebApi (.Net Core).

I already managed to send smaller file as part of Multipart Request like in last post here: link

To send bigger file I need (I think) send this file as StreamContent, however i am getting Content length = 0 in API which receives request.

Problem occurs even when I am sending (for test) smaller files (10 Mb).

Clientside code:

    [HttpPost("UploadFiles")]
    public async Task<IActionResult> Post(IFormFile file)
    {
        var filePath = Path.GetTempFileName();

        using (var stream = new FileStream(filePath, FileMode.Create, FileAccess.ReadWrite))
        {
            await file.CopyToAsync(stream);
            using (var formDataContent = new MultipartFormDataContent())
            {
                using (var httpClient = new HttpClient())
                {
                    formDataContent.Add(CreateFileContent(stream, "myfile.test", "application/octet-stream"));

                    var response = await httpClient.PostAsync(
                        "http://localhost:56595/home/upload",
                        formDataContent);

                    return Json(response);
                }
            }
        }
    }

    internal static StreamContent CreateFileContent(Stream stream, string fileName, string contentType)
    {
        var fileContent = new StreamContent(stream);
        fileContent.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("form-data")
        {
            Name = "\"file\"",
            FileName = "\"" + fileName + "\"",
        };
        fileContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(contentType);
        return fileContent;
    }

Serverside code:

    [HttpPost]
    public ActionResult Upload()
    {
        IFormFile fileFromRequest = Request.Form.Files.First();

        string myFileName = fileFromRequest.Name;

        // some code

        return Ok();
    }

Where is the problem?

To create Multipart request I used advices from:

HttpClient StreamContent append filename twice

POST StreamContent with Multiple Files

Answer 1

Finally I figured it out:

There were 2 problems:

1. Stream pointer position

In client side code, change this:

await file.CopyToAsync(stream);

to that:

await file.CopyToAsync(stream);
stream.Position = 0;

Problem was that file from request was copied to stream and left position of pointer in the end of the stream. That is why request send from client had stream with proper length, but actually when it started to read it, it could not (read 0 bytes).

2. Wrong way of handling request on server.

I used code from dotnetcoretutorials.com

---

Working code below:

Client side:

    [HttpPost("UploadFiles")]
    public async Task<IActionResult> Post(IFormFile file)
    {
        var filePath = Path.GetTempFileName();
        using (var stream = new FileStream(filePath, FileMode.Create, FileAccess.ReadWrite))
        {
            await file.CopyToAsync(stream);
            stream.Position = 0;
            using (var formDataContent = new MultipartFormDataContent())
            {
                using (var httpClient = new HttpClient())
                {
                    formDataContent.Add(CreateFileContent(stream, "myfile.test", "application/octet-stream"));

                    var response = await httpClient.PostAsync(
                        "http://localhost:56595/home/upload",
                        formDataContent);
                    return Json(response);
                }
            }
        }
    }

    internal static StreamContent CreateFileContent(Stream stream, string fileName, string contentType)
    {
        var fileContent = new StreamContent(stream);
        fileContent.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("form-data")
        {
            Name = "\"file\"",
            FileName = "\"" + fileName + "\"",
        };
        fileContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(contentType);
        return fileContent;
    }

Server side:

Controller:

            [HttpPost]
            [DisableFormValueModelBinding]
            public async Task<IActionResult> Upload()
            {
                var viewModel = new MyViewModel();
                try
                {
                    FormValueProvider formModel;
                    using (var stream = System.IO.File.Create("c:\\temp\\myfile.temp"))
                    {
                        formModel = await Request.StreamFile(stream);
                    }

                    var bindingSuccessful = await TryUpdateModelAsync(viewModel, prefix: "",
                        valueProvider: formModel);

                    if (!bindingSuccessful)
                    {
                        if (!ModelState.IsValid)
                        {
                            return BadRequest(ModelState);
                        }
                    }
                }
                catch(Exception exception)
                {
                    throw;
                }
                return Ok(viewModel);
            }

Helper classes for methods from controller:

    public static class MultipartRequestHelper
{
    // Content-Type: multipart/form-data; boundary="----WebKitFormBoundarymx2fSWqWSd0OxQqq"
    // The spec says 70 characters is a reasonable limit.
    public static string GetBoundary(MediaTypeHeaderValue contentType, int lengthLimit)
    {
        var boundary = HeaderUtilities.RemoveQuotes(contentType.Boundary);
        if (string.IsNullOrWhiteSpace(boundary.ToString()))
        {
            throw new InvalidDataException("Missing content-type boundary.");
        }

        if (boundary.Length > lengthLimit)
        {
            throw new InvalidDataException(
                $"Multipart boundary length limit {lengthLimit} exceeded.");
        }

        return boundary.ToString();
    }

    public static bool IsMultipartContentType(string contentType)
    {
        return !string.IsNullOrEmpty(contentType)
               && contentType.IndexOf("multipart/", StringComparison.OrdinalIgnoreCase) >= 0;
    }

    public static bool HasFormDataContentDisposition(ContentDispositionHeaderValue contentDisposition)
    {
        // Content-Disposition: form-data; name="key";
        return contentDisposition != null
               && contentDisposition.DispositionType.Equals("form-data")
               && string.IsNullOrEmpty(contentDisposition.FileName.ToString())
               && string.IsNullOrEmpty(contentDisposition.FileNameStar.ToString());
    }

    public static bool HasFileContentDisposition(ContentDispositionHeaderValue contentDisposition)
    {
        // Content-Disposition: form-data; name="myfile1"; filename="Misc 002.jpg"
        return contentDisposition != null
               && contentDisposition.DispositionType.Equals("form-data")
               && (!string.IsNullOrEmpty(contentDisposition.FileName.ToString())
                   || !string.IsNullOrEmpty(contentDisposition.FileNameStar.ToString()));
    }
}

public static class FileStreamingHelper
{
    private static readonly FormOptions _defaultFormOptions = new FormOptions();

    public static async Task<FormValueProvider> StreamFile(this HttpRequest request, Stream targetStream)
    {
        if (!MultipartRequestHelper.IsMultipartContentType(request.ContentType))
        {
            throw new Exception($"Expected a multipart request, but got {request.ContentType}");
        }

        // Used to accumulate all the form url encoded key value pairs in the 
        // request.
        var formAccumulator = new KeyValueAccumulator();
        string targetFilePath = null;

        var boundary = MultipartRequestHelper.GetBoundary(
            MediaTypeHeaderValue.Parse(request.ContentType),
            _defaultFormOptions.MultipartBoundaryLengthLimit);
        var reader = new MultipartReader(boundary, request.Body);

        var section = await reader.ReadNextSectionAsync();
        while (section != null)
        {
            ContentDispositionHeaderValue contentDisposition;
            var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out contentDisposition);

            if (hasContentDispositionHeader)
            {
                if (MultipartRequestHelper.HasFileContentDisposition(contentDisposition))
                {
                    await section.Body.CopyToAsync(targetStream);
                }
                else if (MultipartRequestHelper.HasFormDataContentDisposition(contentDisposition))
                {
                    // Content-Disposition: form-data; name="key"
                    //
                    // value

                    // Do not limit the key name length here because the 
                    // multipart headers length limit is already in effect.
                    var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name);
                    var encoding = GetEncoding(section);
                    using (var streamReader = new StreamReader(
                        section.Body,
                        encoding,
                        detectEncodingFromByteOrderMarks: true,
                        bufferSize: 1024,
                        leaveOpen: true))
                    {
                        // The value length limit is enforced by MultipartBodyLengthLimit
                        var value = await streamReader.ReadToEndAsync();
                        if (String.Equals(value, "undefined", StringComparison.OrdinalIgnoreCase))
                        {
                            value = String.Empty;
                        }
                        formAccumulator.Append(key.ToString(), value);

                        if (formAccumulator.ValueCount > _defaultFormOptions.ValueCountLimit)
                        {
                            throw new InvalidDataException($"Form key count limit {_defaultFormOptions.ValueCountLimit} exceeded.");
                        }
                    }
                }
            }

            // Drains any remaining section body that has not been consumed and
            // reads the headers for the next section.
            section = await reader.ReadNextSectionAsync();
        }

        // Bind form data to a model
        var formValueProvider = new FormValueProvider(
            BindingSource.Form,
            new FormCollection(formAccumulator.GetResults()),
            CultureInfo.CurrentCulture);

        return formValueProvider;
    }

    private static Encoding GetEncoding(MultipartSection section)
    {
        MediaTypeHeaderValue mediaType;
        var hasMediaTypeHeader = MediaTypeHeaderValue.TryParse(section.ContentType, out mediaType);
        // UTF-7 is insecure and should not be honored. UTF-8 will succeed in 
        // most cases.
        if (!hasMediaTypeHeader || Encoding.UTF7.Equals(mediaType.Encoding))
        {
            return Encoding.UTF8;
        }
        return mediaType.Encoding;
    }
}

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class DisableFormValueModelBindingAttribute : Attribute, IResourceFilter
{
    public void OnResourceExecuting(ResourceExecutingContext context)
    {
        var formValueProviderFactory = context.ValueProviderFactories
            .OfType<FormValueProviderFactory>()
            .FirstOrDefault();
        if (formValueProviderFactory != null)
        {
            context.ValueProviderFactories.Remove(formValueProviderFactory);
        }

        var jqueryFormValueProviderFactory = context.ValueProviderFactories
            .OfType<JQueryFormValueProviderFactory>()
            .FirstOrDefault();
        if (jqueryFormValueProviderFactory != null)
        {
            context.ValueProviderFactories.Remove(jqueryFormValueProviderFactory);
        }
    }

    public void OnResourceExecuted(ResourceExecutedContext context)
    {
    }
}

Additional thoughts:

• (on clientside) line:

  fileContent.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(contentType);

is not necessary to send the file.

• (on clientside) file is send when MediaTypeHeaderValue is one of these:

  application/x-msdownload

  application/json

  application/octet-stream

• (on serverside) to use lines with contentDisposition.FileNameStar on serverside you need to change them to contentDisposition.FileNameStar.ToString()

• (on serverside) code used in question for serverside will work with smaller files (Mb's) but to send GB file we need code which is pasted in the answer.

• some parts of code are taken from aspnet core docs