How do I force release memory occupied by MemoryStream?

Question

I have the following code:

const int bufferSize = 1024 * 1024;
var buffer = new byte[bufferSize];
for (int i = 0; i < 10; i++)
{
    const int writesCount = 400;
    using (var stream = new MemoryStream(writesCount * bufferSize))
    {
        for (int j = 0; j < writesCount; j++)
        {
            stream.Write(buffer, 0, buffer.Length);
        }
        stream.Close();
    }
}

which I run on a 32-bit machine.

The first iteration finishes just fine and then on the next iteration I get a System.OutOfMemoryException exception on the line that news the MemoryStream.

Why isn't the previous MemoryStream memory reclaimed despite using statement? How do I force release of memory used by the MemoryStream?

Answer 1

I don't think the problem is the garbage collector not doing its job. If the GC is under memory pressure it should run and reclaim the 400 MBs you've just allocated.

This is more likely down to the GC not finding a contigious 400 MB block.

Rather, an “out of memory” error happens because the process is unable to find a large enough section of contiguous unused pages in its virtual address space to do the requested mapping.

You should read Eric Lippert's blog entry "Out Of Memory" Does Not Refer to Physical Memory

You're far better off doing both of the below.

1. Reusing the memory block you've allocated (why are you creating another with the exact same size)
2. Allocating much smaller chunks (less than 85KBs)

Prior to Dotnet 4.5, Dotnet constructed two heaps, Small Object Heap (SOH) and Large Object Heap (LOH). See Large Object Hearp Improvements in .NET 4.5 by Brandon Bray. Your MemoryStream is being allocated in LOH, and not compacted (defragmented) for the duration of the process, making it much more likely that multiple calls to allocate this large amount of memory will throw an OutOfMemoryException

The CLR manages two different heaps for allocation, the small object heap (SOH) and the large object heap (LOH). Any allocation greater than or equal to 85,000 bytes goes on the LOH. Copying large objects has a performance penalty, so the LOH is not compacted unlike the SOH. Another defining characteristic is that the LOH is only collected during a generation 2 collection. Together, these have the built-in assumption that large object allocations are infrequent.

Answer 2

Looks like you're allocating too much than your system can handle. Your code runs fine on my machine, but if I change it like this :

const int bufferSize = 1024 * 1024 * 2;

I get the same error as you.

But if I change the target processor to x64, then the code runs, which seems logical as you can address lot more memory.

Detailed explanation on this article : http://www.guylangston.net/blog/Article/MaxMemory And some information on this question : Maximum Memory a .NET process can allocate