Uploading and Downloading large files in ASP.NET Core 3.1?

Question

I am working on an ASP.NET Core 3.1 API project using clean architecture and I have the following classlibs (tiers):

• Infrastructure (security stuff and upload helpers etc ...)
• Persistence (DA layer)
• Domain (Domain Models)
• Application (use cases - Business logics)
• API (API Project as my startup project)

I want to be able to upload large files to server (like 2Gb of file size or even more) and download them after that and want to do it without having future problems with memory overflowing and everything.

Any help would be appreciated.

Answer 1

If you have files that large, never use byte[] or MemoryStream in your code. Only operate on streams if you download/upload files.

You have a couple of options:

• If you control both client and server, consider using something like tus. There are both client- and server-implementations for .NET. This would probably the easiest and most robust option.
• If you upload large files with the HttpClient, simply use the StreamContent class to send them. Again, don't use a MemoryStream as source, but something else like a FileStream.
• If you download large files with the HttpClient, it is important to specify the HttpCompletionOptions, for example var response = await httpClient.SendAsync(httpRequest, HttpCompletionOption.ResponseHeadersRead). Otherwise, the HttpClient would buffer the entire response in memory. You can then process the response file as a stream via var stream = response.Content.ReadAsStreamAsync().

ASP.NET Core specific advice:

• If you want to receive files via HTTP POST, you need to increase the request size limit: [RequestSizeLimit(10L * 1024L * 1024L * 1024L)] and [RequestFormLimits(MultipartBodyLengthLimit = 10L * 1024L * 1024L * 1024L)]. In addition, you need to disable the form value binding, otherwise the whole request will be buffered into memory:

   [AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]    public class DisableFormValueModelBindingAttribute : Attribute, IResourceFilter    {        public void OnResourceExecuting(ResourceExecutingContext context)        {            var factories = context.ValueProviderFactories;            factories.RemoveType<FormValueProviderFactory>();            factories.RemoveType<FormFileValueProviderFactory>();            factories.RemoveType<JQueryFormValueProviderFactory>();        }         public void OnResourceExecuted(ResourceExecutedContext context)        {        }    } 

• To return a file from a controller, simple return it via the File method, which accepts a stream: return File(stream, mimeType, fileName);

---

A sample controller would look like this (see https://docs.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.1 for the missing helper classes):

private const MaxFileSize = 10L * 1024L * 1024L * 1024L; // 10GB, adjust to your need  [DisableFormValueModelBinding] [RequestSizeLimit(MaxFileSize)] [RequestFormLimits(MultipartBodyLengthLimit = MaxFileSize)] public async Task ReceiveFile() {     if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))         throw new BadRequestException("Not a multipart request");      var boundary = MultipartRequestHelper.GetBoundary(MediaTypeHeaderValue.Parse(Request.ContentType));     var reader = new MultipartReader(boundary, Request.Body);      // note: this is for a single file, you could also process multiple files     var section = await reader.ReadNextSectionAsync();      if (section == null)         throw new BadRequestException("No sections in multipart defined");      if (!ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out var contentDisposition))         throw new BadRequestException("No content disposition in multipart defined");      var fileName = contentDisposition.FileNameStar.ToString();     if (string.IsNullOrEmpty(fileName))     {         fileName = contentDisposition.FileName.ToString();     }      if (string.IsNullOrEmpty(fileName))         throw new BadRequestException("No filename defined.");      using var fileStream = section.Body;     await SendFileSomewhere(fileStream); }  // This should probably not be inside the controller class private async Task SendFileSomewhere(Stream stream) {     using var request = new HttpRequestMessage()     {         Method = HttpMethod.Post,         RequestUri = new Uri("YOUR_DESTINATION_URI"),         Content = new StreamContent(stream),     };     using var response = await _httpClient.SendAsync(request);     // TODO check response status etc. }  

In this example, we stream the entire file to another service. In some cases, it would be better to save the file temporarily to the disk.