Where predicates does not releases memory

Question

If I include outside reference inside where predicate then memory does not get releases.

Let's say I have a List<object> then if I write a where predicate like this :

    List<object> myList = new List<object>();
    ...
    myList.add(object);
    ...

    Expression<Func<object,bool>> predicate = p => myList.Contains(p);

Even If I make myList = null or predicate = null, it is not releasing memory.

I have List<object> itemsource binded to DataGrid. I also make it's ItemSource null, disposing DataGrid, DataGrid null. . I also have analyzed this issue with ANTS Memory Profiler 7.4. It also shows me that because of wherepredicate it is holding reference.

If I change my wherepredicate like this in dispose(), then memory is getting released.

    Expression<Func<object,bool>> predicate = p => p.id == 0;

that means removing reference in WherePredicate.

Answer 1

Mmmh... interesting... even Expression<> cause closure... I didn't know...

the end result: predicate doesn't have a reference to myList

I'll explain:

private static bool IsDebug()
{
    // Taken from http://stackoverflow.com/questions/2104099/c-sharp-if-then-directives-for-debug-vs-release
    object[] customAttributes = Assembly.GetExecutingAssembly().GetCustomAttributes(typeof(DebuggableAttribute), false);

    if ((customAttributes != null) && (customAttributes.Length == 1))
    {
        DebuggableAttribute attribute = customAttributes[0] as DebuggableAttribute;
        return (attribute.IsJITOptimizerDisabled && attribute.IsJITTrackingEnabled);
    }

    return false;
}

static void Main(string[] args)
{
    // Check x86 or x64
    Console.WriteLine(IntPtr.Size == 4 ? "x86" : "x64");

    // Check Debug/Release
    Console.WriteLine(IsDebug() ? "Debug, USELESS BENCHMARK" : "Release");

    // Check if debugger is attached
    Console.WriteLine(System.Diagnostics.Debugger.IsAttached ? "Debugger attached, USELESS BENCHMARK!" : "Debugger not attached");

    Console.WriteLine();

    {
        long memory = GC.GetTotalMemory(true);

        // A big array, big enough that we can see its allocation in
        // memory
        byte[] buffer = new byte[10000000];

        Console.WriteLine("Just allocated the array: {0}", GC.GetTotalMemory(true) - memory);

        // A List<>, containing a reference to the buffer
        List<object> myList = new List<object>();
        myList.Add(buffer);

        Console.WriteLine("Added to the List<>: {0}", GC.GetTotalMemory(true) - memory);

        // We want to be sure that buffer is referenced at least up to
        // this point
        GC.KeepAlive(buffer);

        // But clearly setting buffer = null is useless, because the
        // List<> has anothe reference
        buffer = null;
        Console.WriteLine("buffer = null: {0}", GC.GetTotalMemory(true) - memory);

        // If I Clear() the List<>, the last reference to the buffer
        // is removed, and now the buffer can be freed
        myList.Clear();
        Console.WriteLine("myList.Clear(): {0}", GC.GetTotalMemory(true) - memory);

        GC.KeepAlive(myList);
    }

    Console.WriteLine();
    GC.Collect();

    {
        long memory = GC.GetTotalMemory(true);

        // A big array, big enough that we can see its allocation in
        // memory
        byte[] buffer = new byte[10000000];

        Console.WriteLine("Just allocated the array: {0}", GC.GetTotalMemory(true) - memory);

        // A List<>, containing a reference to the buffer
        List<object> myList = new List<object>();
        myList.Add(buffer);

        Console.WriteLine("Added to the List<>: {0}", GC.GetTotalMemory(true) - memory);

        // We want to be sure that buffer is referenced at least up to
        // this point
        GC.KeepAlive(buffer);

        // But clearly setting buffer = null is useless, because the
        // List<> has another reference
        buffer = null;
        Console.WriteLine("buffer = null: {0}", GC.GetTotalMemory(true) - memory);

        // We want to be sure that the List<> is referenced at least
        // up to this point
        GC.KeepAlive(myList);

        // If I set to null myList, the last reference to myList
        // and to buffer are removed
        myList = null;
        Console.WriteLine("myList = null: {0}", GC.GetTotalMemory(true) - memory);
    }

    Console.WriteLine();
    GC.Collect();

    {
        long memory = GC.GetTotalMemory(true);

        // A big array, big enough that we can see its allocation in
        // memory
        byte[] buffer = new byte[10000000];

        Console.WriteLine("Just allocated the array: {0}", GC.GetTotalMemory(true) - memory);

        // A List<>, containing a reference to the buffer
        List<object> myList = new List<object>();
        myList.Add(buffer);

        Console.WriteLine("Added to the List<>: {0}", GC.GetTotalMemory(true) - memory);

        // A predicate, containing a reference to myList
        Expression<Func<object, bool>> predicate1 = p => myList.Contains(p);
        Console.WriteLine("Created a predicate p => myList.Contains(p): {0}", GC.GetTotalMemory(true) - memory);

        // A second predicate, **not** containing a reference to
        // myList
        Expression<Func<object, bool>> predicate2 = p => p.GetHashCode() == 0;
        Console.WriteLine("Created a predicate p => p.GetHashCode() == 0: {0}", GC.GetTotalMemory(true) - memory);

        // We want to be sure that buffer is referenced at least up to
        // this point
        GC.KeepAlive(buffer);

        // But clearly setting buffer = null is useless, because the
        // List<> has another reference
        buffer = null;
        Console.WriteLine("buffer = null: {0}", GC.GetTotalMemory(true) - memory);

        // We want to be sure that the List<> is referenced at least
        // up to this point
        GC.KeepAlive(myList);

        // If I set to null myList, an interesting thing happens: the
        // memory is freed, even if the predicate1 is still alive!
        myList = null;
        Console.WriteLine("myList = null: {0}", GC.GetTotalMemory(true) - memory);

        // We want to be sure that the predicates are referenced at 
        // least up to this point
        GC.KeepAlive(predicate1);
        GC.KeepAlive(predicate2);

        try
        {
            // We compile the predicate1
            Func<object, bool> fn = predicate1.Compile();
            // And execute it!
            fn(5);
        }
        catch (NullReferenceException)
        {
            Console.WriteLine("predicate1 is 'pointing' to a null myList");
        }
    }
}

This is a sample test in three parts: the basic point is that a big byte[] array is allocated, and through checking how much memory is allocated we check if the array is still allocated in some way. it is very important that this code is executed in Release mode without the debugger (CTRL+F5). If you don't do it, you'll get a warning when the program starts

The first two "parts" are only to show that a List<> does keep "alive" the items it references (so the byte[] in this case), and freeing the List<> or .Clear()ing it lets the GC collect the byte[].

The third part is more interesting: there are both a List<> and an Expression<>... Both seems to keep a reference to the byte[], but this is an illusion. The Expression<> as written causes the compiler to generate a "closure" around the myList<> variable. Using ILSpy it is quite easy to see:

Program.<>c__DisplayClassb <>c__DisplayClassb = new Program.<>c__DisplayClassb();
<>c__DisplayClassb.myList = new List<object>();
<>c__DisplayClassb.myList.Add(buffer3);

ParameterExpression parameterExpression = Expression.Parameter(typeof(object), "p");
Expression<Func<object, bool>> predicate = Expression.Lambda<Func<object, bool>>(Expression.Call(Expression.Field(Expression.Constant(<>c__DisplayClassb), fieldof(Program.<>c__DisplayClassb.myList)), methodof(List<object>.Contains(!0)), new Expression[]
{
    parameterExpression
}), new ParameterExpression[]
{
    parameterExpression
});

(if you don't have ILSpy, you can take a look at the code generated by the online compiler TryRoslyn for a simpler sample)

An hidden class <>c__DisplayClassb is generated by the compiler, with a field myList. So instead of having a "local" variable myList, the method has as a local variable <>c__DisplayClassb that has a field myList. The predicate1 doesn't directly keep a reference to myList, but has a reference to the variable <>c__DisplayClassb (see the Expression.Constant(<>c__DisplayClassb)?), so when

<>c__DisplayClassb.myList = null;

the predicate1 does still has a reference to <>c__DisplayClassb, but the <>c__DisplayClassb.myList is null, so there are no more references to myList.