.NET Memory issues loading ~40 images, memory not reclaimed, potentially due to LOH fragmentation

Question

Well, this is my first foray into memory profiling a .NET app (CPU tuning I have done) and I am hitting a bit of a wall here.

I have a view in my app which loads 40 images (max) per page, each running about ~3MB. The max number of pages is 10. Seeing as I don't want to keep 400 images or 1.2GB in memory at once, I set each image to null when the page is changed.

Now, at first I thought that I must just have stale references to these images. I downloaded ANTS profiler (great tool BTW) and ran a few tests. The object lifetime graph tells me that I don't have any references to these images other than the single reference in the parent class (which is by design, also confirmed by meticulously combing through my code):

The parent class SlideViewModelBase sticks around forever in a cache, but the MacroImage property is set to null when the page is changed. I don't see any indication that these objects should be kept around longer than expected.

I next took a look at the large object heap and memory usage in general. After looking at three pages of images I have 691.9MB of unmanaged memory allocated and 442.3MB on the LOH. System.Byte[], which comes from my System.Drawing.Bitmap to BitmapImage conversion is taking pretty much all of the LOH space. Here is my conversion code:

public static BitmapSource ToBmpSrc( this Bitmap b )
{
    var bi = new BitmapImage();
    var ms = new MemoryStream();
    bi.CacheOption = BitmapCacheOption.OnLoad;
    b.Save( ms,  ImageFormat.Bmp );
    ms.Position = 0;
    bi.BeginInit();
    ms.Seek( 0, SeekOrigin.Begin );
    bi.StreamSource = ms;
    bi.EndInit();
    return bi;
}

I am having a hard time finding where all of that unmanaged memory is going. I suspected the System.Drawing.Bitmap objects at first, but ANTS doesn't show them sticking around, and I also ran a test where I made absolutely sure that all of them were disposed and it didn't make a difference. So I haven't yet figured out where all of that unmanaged memory is coming from.

My two current theories are:

1. LOH fragmentation. If I navigate away from the paged view and click a couple of buttons about half of the ~1.5GB is reclaimed. Still too much, but interesting nonetheless.
2. Some weird WPF binding thing. We do use databinding to display these images and I am no expert in regards to the ins and outs of how these WPF controls work.

If anyone has any theories or profiling tips I would be extremely grateful as (of course) we are on a tight deadline and I am scrambling a bit to get this final part done and working. I think I've been spoiled by tracking down memory leaks in C++ ... who woulda' thought?

If you need more info or would like me to try something else please ask. Sorry about the wall-o-text here, I tried to keep it as concise as possible.

Answer 1

This blog post appears to descibe what you are seeing, and the proposed solution was to create an implementation of Stream that wraps another stream.

The Dispose method of this wrapper class needs to release the wrapped stream, so that it can be garbage collected. Once the BitmapImage is initialised with this wrapper stream, the wrapper stream can be disposed, releasing the underlying stream, and allowing the large byte array itself to be freed.

The BitmapImage keeps a reference to the source stream so it keeps the MemoryStream object alive. Unfortunately, even though MemoryStream.Dispose has been invoked, it doesn't release the byte array that the memory stream wraps. So, in this case, bitmap is referencing stream, which is referencing buffer, which may be taking up a lot of space on the large object heap. There isn't a true memory leak; when there are no more references to bitmap, all these objects will (eventually) be garbage collected. But since bitmap has already made its own private copy of the image (for rendering), it seems rather wasteful to have the now-unnecessary original copy of the bitmap still in memory.

Also, what version of .NET are you using? Prior to .NET 3.5 SP1, there was a known issue where a BitmapImage could cause a memory leak. The workaround was to call Freeze on the BitmapImage.

Answer 2

Where are you closing and disposing the memory stream? It could be that the GC has to work a lot harder to free the resources by moving the up several generations before executing the destructors on the object (which generally call the dispose if you forgot to do it).

In your case, you can't dispose the memory stream until you are done with the image. When you want them unloaded, loop through the images and try disposing the memory stream.