Why doesn't object with finalizer get collected even if it is unrooted?

Question

I've encountered an issue with finalizable objects that doesn't get collected by GC if Dispose() wasn't called explicitly. I know that I should call Dispose() explicitly if an object implements IDisposable, but I always thought that it is safe to rely upon framework and when an object becomes unreferenced it can be collected.

But after some experiments with windbg/sos/sosex I've found that if GC.SuppressFinalize() wasn't called for finalizable object it doesn't get collected, even if it becomes unrooted. So, if you extensively use finalizable objects(DbConnection, FileStream, etc) and not disposing them explicitly you can encounter too high memory consumption or even OutOfMemoryException.

Here is a sample application:

public class MemoryTest
{
    private HundredMegabyte hundred;

    public void Run()
    {
        Console.WriteLine("ready to attach");
        for (var i = 0; i < 100; i++)
        {
            Console.WriteLine("iteration #{0}", i + 1);
            hundred = new HundredMegabyte();
            Console.WriteLine("{0} object was initialized", hundred);
            Console.ReadKey();
            //hundred.Dispose();
            hundred = null;
        }
    }

    static void Main()
    {
        var test = new MemoryTest();
        test.Run();
    }
}

public class HundredMegabyte : IDisposable
{
    private readonly Megabyte[] megabytes = new Megabyte[100];

    public HundredMegabyte()
    {
        for (var i = 0; i < megabytes.Length; i++)
        {
            megabytes[i] = new Megabyte();
        }
    }

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    ~HundredMegabyte()
    {
        Dispose(false);
    }

    private void Dispose(bool disposing)
    {
    }

    public override string ToString()
    {
        return String.Format("{0}MB", megabytes.Length);
    }
}

public class Megabyte
{
    private readonly Kilobyte[] kilobytes = new Kilobyte[1024];

    public Megabyte()
    {
        for (var i = 0; i < kilobytes.Length; i++)
        {
            kilobytes[i] = new Kilobyte();
        }
    }
}

public class Kilobyte
{
    private byte[] bytes = new byte[1024];
}

Even after 10 iterations you can find that memory consumption is too high(from 700MB to 1GB) and gets even higher with more iterations. After attaching to process with WinDBG you can find that all large objects are unrooted, but not collected.

Situation changes if you call SuppressFinalize() explicitly: memory consumption is stable around 300-400MB even under high pressure and WinDBG shows that there are no unrooted objects, memory is free.

So the question is: Is it a bug in framework? Is there any logical explanation?

More details:

After each iteration, windbg shows that:

• finalization queue is empty
• freachable queue is empty
• generation 2 contains objects(Hundred) from previous iterations
• objects from previous iterations are unrooted

Answer 1

An object with a finalizer doesn't behave the same way as an object lacking one.

When a GC occurs, and SuppressFinalize has not been called, the GC won't be able to collect the instance, because it must execute the Finalizer. Therefore, the finalizer is executed, AND the object instance is promoted to generation 1 (object which survived a first GC), even if it's already without any living reference.

Generation 1 (and Gen2) objects are considered long lived, and will be considered for garbage collection only if a Gen1 GC isn't sufficient to free enough memory. I think that during your test, Gen1 GC is always sufficient.

This behavior has an impact on GC performance, as it negates the optimisation brought by having several générations (you have short duration objects in the gen1 ).

Essentially, having a Finalizer and failing to prevent the GC from calling it will always promote already dead objects to the long lived heap, which isn't a good thing.

You should therefore Dispose properly your IDisposable objects AND avoid Finalizers if not necessary (and if necessary, implement IDisposable and call GC.SuppressFinalize. )

Edit: I didn't read the code example well enough: Your data looks like it is meant to reside in the Large Object Heap (LOH), but in fact isn't: You have a lot of small arrays of references containing at the end of the tree small arrays of bytes.

Putting short duration object in the LOH is even worse, as they won't be compacted... And therefore you could run OutOfMemory with lot of free memory, if the CLR isn't able to find an empty memory segment long enough to contains a large chunk of data.

Answer 2

I think the idea behind this is, when you implement IDisposable, it's because you are handling unmanaged resources, and require to dispose your resource manually.

If the GC was to call Dispose or try to get rid of it, it would flush the unmanaged stuff too, which can very well be used somewhere else, the GC has no way to know that.

If the GC was to removed unrooted object, you would lose reference toward unmanaged resource which would lead to memory leaks.

So... You're managed, or your not. There is simply no good way for the GC to handle not-disposed IDisposables.