Asp.Net Core API Controller method returning IAsyncEnumerable<T> not producing the intended behavior

Question

I have an API Controller setup which return 277,000+ items:

[HttpPost]
public async IAsyncEnumerable<LocationDto> GetLocations([FromBody] LocationReportQueryDto locationReportQuery, CancellationToken token = default)
{
    var result = await locationReportDataAccess.GetFilteredLocationsAsync(locationReportQuery, token);

    foreach (var location in result)
    {
        yield return location;
    }
}

and instead of streaming each location back, Asp.Net is actually buffering the response.

Client side code:

public async Task<List<ItemLocDto>> GetFilteredLocationsAsync(LocationReportQueryDto locationReportQuery, CancellationToken token = default)
{
    var httpClient = httpClientFactory.CreateClient("DataAccess");
    var response = await httpClient.PostAsJsonAsync("/reports/LocationReport/GetFilteredLocations", locationReportQuery, token);
    response.EnsureSuccessStatusCode();
    var list = await response.Content.ReadFromJsonAsync<List<ItemLocDto>>(cancellationToken: token);
    if (list == null) throw new HttpRequestException("LocationReportDataAccess GetFilteredLocationsAsync HTTP Call - Response is null");
    return list;
}

I'm getting this exception upon returning from this method:

blazor.server.js:1 [2022-12-18T04:51:19.544Z] Error: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at System.IO.MemoryStream.set_Capacity(Int32 value)
   at System.IO.MemoryStream.EnsureCapacity(Int32 value)
   at System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Http.HttpContent.LimitMemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.MemoryStream.WriteAsync(ReadOnlyMemory`1 buffer, CancellationToken cancellationToken)
--- End of stack trace from previous location ---
   at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.CopyToAsyncCore(Stream destination, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionResponseContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken)
   at System.Net.Http.HttpContent.LoadIntoBufferAsyncCore(Task serializeToStreamTask, MemoryStream tempBuffer)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Portal.Infrastructure.DataAccess.Reports.LocationReportDataAccess.GetFilteredLocationsAsync(LocationReportQueryDto locationReportQuery, CancellationToken token) in C:\Users\LOFT\RiderProjects\Portal\src\Libraries\Portal.Infrastructure\DataAccess\Reports\LocationReportDataAccess.cs:line 30
   at Portal.Services.Reporting.LocationResultsService.GetResultsAsync(LocationInputConfig config, CancellationToken token) in C:\Users\LOFT\RiderProjects\Portal\src\Libraries\Portal.Services\Reporting\LocationResultsService.cs:line 137
   at Portal.Web.Pages.Reports.LocationList.OnGenerateClick() in C:\Users\LOFT\RiderProjects\Portal\src\Presentation\Portal.Web\Pages\Reports\LocationList.razor:line 679
   at Microsoft.AspNetCore.Components.ComponentBase.CallStateHasChangedOnAsyncCompletion(Task task)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.GetErrorHandledTask(Task taskToHandle, ComponentState owningComponentState)

I cannot use pagination because I'm using a legacy database, so I'm trying to stream all of the data from the API endpoint.

The "result" is of type List<LocationDto,> could this be the problem, or do I need to reduce a buffer size somewhere?

Answer 1

Your exception stack trace shows that you're getting the OOM from within GetFilteredLocationsAsync. This is because it returns a List<ItemLocDto>, presumably reading the entire response into memory as JSON and then parsing them into ItemLocDto objects while building/extending a List<T> to contain those objects. It's the GetFilteredLocationsAsync that is buffering into memory.

So, you'll need to change something about how /reports/LocationReport/GetFilteredLocations is called. If it can stream results, then you may be able to stream from the source to GetLocations without buffering in-memory as a List<T>. Otherwise, you'll probably need some sort of pagination on both APIs.