OutOfMemoryException when creating huge string in ASP.NET

Question

When exporting a lot of data to a string (csv format), I get a OutOfMemoryException. What's the best way to tackle this? The string is returned to a Flex Application.

What I'd do is export the csv to the server disk and give back an url to Flex. Like this, I can flush the stream writing to the disk.

Update:

String is build with a StringBuilder:

StringBuilder stringbuilder = new StringBuilder();
string delimiter = ";";
bool showUserData = true;

// Get the data from the sessionwarehouse
List<DwhSessionDto> collection 
     =_dwhSessionRepository.GetByTreeStructureId(treeStructureId);

// ADD THE HEADERS
stringbuilder.Append("UserId" + delimiter);
if (showUserData)
{
    stringbuilder.Append("FirstName" + delimiter);
    stringbuilder.Append("LastName" + delimiter);
}
stringbuilder.Append("SessionId" + delimiter);
stringbuilder.Append("TreeStructureId" + delimiter);
stringbuilder.Append("Name" + delimiter);
stringbuilder.Append("Score" + delimiter);
stringbuilder.Append("MaximumScore" + delimiter);
stringbuilder.Append("MinimumScore" + delimiter);
stringbuilder.Append("ReducedScore" + delimiter);
stringbuilder.Append("ReducedMaximumScore" + delimiter);
stringbuilder.Append("Duration" + delimiter);
stringbuilder.AppendLine("Category" + delimiter);

foreach (var dwhSessionDto in collection)
{
    stringbuilder.Append(
        getPackageItemsInCsvFromDwhSessionDto(
            dwhSessionDto, delimiter, showUserData));
}

return stringbuilder.ToString();

The string is sent back to Flex like this:

var contentType = "text/csv";
string result = exportSessionService.ExportPackage(treeStructureId);
// Write the export to the response
_context.Response.ContentType = contentType;
_context.Response.AddHeader("content-disposition", 
    String.Format("attachment; filename={0}", treeStructureId + ".csv"));

// do not Omit the Vary star so the caching at the client side will be disabled
_context.Response.Cache.SetOmitVaryStar(false);
_context.Response.Cache.SetCacheability(HttpCacheability.NoCache);

if (!String.IsNullOrEmpty(result))
{
    _context.Response.Output.Write(result);
    _context.Response.Output.Close();
}
else
{
    _context.Response.Output.Write("No logs");
}

Answer 1

Your CSV strings are growing to over 80000 bytes and ending up on the LargeObjectHeap. The LOH is not garbage collected in the same way as other generations and can fragment over time, such as after many requests to your server or if you use string concatenation (naughty!) to build this csv data. The result is that your program reserves much more memory than it's actually using and an OutOfMemory exception is thrown.

The fix in this instance is to write your csv data directly to the Response stream rather than string variables or even StringBuilder. Not only will this avoid the Large Object Heap, but it will keep your overall memory use lower and start pushing data out to your user faster.