Returning a large collection serialized as JSON via MVC controller

Question

I have a very large result set which I want to return to an Ajax call using JSON.

I started with creating a collection of objects and then serialize the collection but the collection creation would throw a System.OutOfMemoryException.

I've now tried to change the implementation to stream the JSON without having a collection but I still get the System.OutOfMemoryException.

Here my current code snippets.

  using (var stream = new MemoryStream())
  {
      using (var streamWriter = new StreamWriter(stream))
      {
          using (var jsonWriter = new JsonTextWriter(streamWriter))
          {
              var serializer = new JsonSerializer();

              serializer.Serialize(jsonWriter,new { pins = MakePins(model), missingLocations = 0 });

              jsonWriter.Flush();
          }
      }

  stream.Seek(0, SeekOrigin.Begin);

  return new FileStreamResult(stream, "application/json");

The MakePins function looks like this:

var pinData = _geographyService.GetEnumerationQueryable()
                .SelectMany(x => x.EnumeratedPersonRoleCollection)
                .ApplyFilter(model).Where(x => x.EnumerationCentre.Location != null)
                .AsNoTracking()
                .AsEnumerable();

return pinData.Select(item => new MapPin
    {
        Id = item.EnumerationCentre.EnumerationCentreUid.ToString(),
        Name = item.Person.FullName,
        FillColour = GetMapPinColour(item, model),
        Latitude = item.EnumerationCentre.Location.Latitude,
        Longitude = item.EnumerationCentre.Location.Longitude,
        Count = item.IssuedVoucherCollection.Count()
    });

I've tried using a yield return instead of the select but the OutOfMemoryException is throw withing the Select function.

I've done a fair bit of googling but can't quite see what else I could try.

Answer 1

Your current solution still have the same problem, because just before return you collect and store all data in the memory stream

You can try something in the following fashion:

public ActionResult RobotsText() {
    Response.ContentType = "application/json";

    Response.Write("[");

    foreach(var item in Items)
    {
         Response.Write(JsonSerializer.Serialize(item));

        if ( /*not last*/)
        {
            Response.Write(",");
        }
    }

    Response.Write("]");   

    return new EmptyResult();
}