Foreach loop using Expression trees

Question

I have seen this Issue while building dynamic Expression Tree and Expression/Statement trees and since I am new to expression trees I am still struggling to understand how to achieve what I want.

A contrived object is below

    public class TestObject
    {
        public TestObject()
        {
            ClassList = new List<Class>();
        }
        public int Age { get; set; }
        public List<Class> ClassList { get; set; } 
    }

    public class Class
    {
        public string Name { get; set; }
        public int ClassId { get; set; }
    }

At run time I iterate through each of the properties and generate a Delegate which does a conversion to string of that property. I have got all that working. The issue I have to deal with now is that for the List type, I need to be able to apply a set of actions to each item in the ClassList property so I need a foreach which allows me to do that.

I currently have this

//type==TestObject at runtime
//propertyName == "ClassList"
   ParameterExpression recordExpression = Expression.Parameter(type, "record");

   memberExpression = MemberExpression.Property(recordExpression, propertyName);

   Type getEnumerableDelegateType =
                typeof(Func<,>).MakeGenericType(new Type[] { type, memberExpression.Type}); 

   var getList = Expression.Lambda(getEnumerableDelegateType, memberExpression, recordExpression);

GetList when compiled and invoked returns the List as expected. What I m struggling with is how to create an expression which will use the result from the lambda expression and iterate over it applying the set of actions I have already created for each Class item.

Ultimately I am looking for a lambda signature to match the overallAction signature below

   var getListFunc = new Func<TestObject, List<Class>>((TestObject obj1) => obj1.ClassList);

   Action<List<Class>> listAction = delegate(List<Class> data)
                {
                    foreach (var dataChannelWithUnitse in data)
                    {
                        //Apply generated delegate
                    }
                };

     Action<TestObject> overallAction = delegate(TestObject data)
                {
                    var x = getListFunc.Invoke(data);
                    listAction.Invoke(x as List<Class>);
                };

Any help is appreciated to help me understand how to do this.

I have currently got this which is exceptioning with variable 'Input' of type 'TestObject' referenced from scope '', but it is not defined

    var typeParam = Expression.Parameter(type, "Input");
    var listVariable = Expression.Variable(memberExpression.Type, "List");
    var enumerator = Expression.Variable(typeof(IEnumerator<>).MakeGenericType(dataType));


    var enumeratorType = typeof(IEnumerator<>).MakeGenericType(dataType);
    var enumerableType = typeof(IEnumerable<>).MakeGenericType(dataType);
    var enumerableParam = Expression.Parameter(enumerableType, "ExtractedCollection");

    var getEnumeratorFunc = Expression.Call(enumerableParam, enumerableType.GetMethod("GetEnumerator"));
    var getEnumeratorLambda = Expression.Lambda(getEnumeratorFunc, enumerableParam);

    var t1 = Expression.Assign(listVariable, Expression.Invoke(getListLambda, typeParam));
    var t2 = Expression.Assign(enumerator, Expression.Invoke(getEnumeratorLambda, listVariable));


    var @break = Expression.Label();

    var funcBlock = Expression.Block(
        new ParameterExpression[] { listVariable, enumerator},

   t1,
   t2,

    Expression.Loop(
        Expression.IfThenElse(

            Expression.NotEqual(Expression.Call(enumerator,typeof(IEnumerator).GetMethod("MoveNext")),Expression.Constant(false)),
                                Expression.Invoke(enumerableExpressions[0],Expression.Property(enumerator, "Current")),

                      Expression.Break(@break))
            , @break), typeParam);



    Expression<Action<TestObject>> lm = Expression.Lambda<Action<TestObject>>(funcBlock,recordExpression);
    var d = lm.Compile(); **//this is exceptioning with " variable 'Input' of type 'TestObject' referenced from scope '', but it is not defined**

Answer 1

relatively_random's solution is great but foreach handles several other scenarios. Check these links to SharpLab to verify what is generated in each of them:

• When the enumerable is a IEnumerable<T>, it checks if the enumerator is null before calling Dispose().
• When the enumerable is not an interface, the enumerator changes to have the type returned by GetEnumerator(). The enumerator is cast to IDisposable before calling Dispose().
• When the enumerator doesn't implement IDisposable, as is used on the check if the enumerator implements IDisposable. (???)
• When the enumerator is a value type, the check for null goes away.
• When the enumerator is a value type and doesn't implement IDisposable, the try/finally goes away.

The use of the type returned by GetEnumerator() is very important so that value type enumerators are not boxed. All the collections in System.Collections.Generic have value type enumerator because calls to its methods are not virtual, resulting in a lot better performance.

Putting all together results in the following code:

static partial class ExpressionEx
{
    public static Expression ForEach<TSource>(Expression enumerable, Expression loopContent)
    {
        var enumerableType = enumerable.Type;
        var getEnumerator = enumerableType.GetMethod("GetEnumerator");
        if (getEnumerator is null)
            getEnumerator = typeof(IEnumerable<>).MakeGenericType(typeof(TSource)).GetMethod("GetEnumerator");
        var enumeratorType = getEnumerator.ReturnType;
        var enumerator = Expression.Variable(enumeratorType, "enumerator");

        return Expression.Block(new[] { enumerator },
            Expression.Assign(enumerator, Expression.Call(enumerable, getEnumerator)),
            EnumerationLoop(enumerator, loopContent));
    }

    public static Expression ForEach<TSource>(Expression enumerable, ParameterExpression loopVar, Expression loopContent)
    {
        var enumerableType = enumerable.Type;
        var getEnumerator = enumerableType.GetMethod("GetEnumerator");
        if (getEnumerator is null)
            getEnumerator = typeof(IEnumerable<>).MakeGenericType(typeof(TSource)).GetMethod("GetEnumerator");
        var enumeratorType = getEnumerator.ReturnType;
        var enumerator = Expression.Variable(enumeratorType, "enumerator");

        return Expression.Block(new[] { enumerator },
            Expression.Assign(enumerator, Expression.Call(enumerable, getEnumerator)),
            EnumerationLoop(enumerator,
                Expression.Block(new[] { loopVar },
                    Expression.Assign(loopVar, Expression.Property(enumerator, "Current")),
                    loopContent)));
    }

    static Expression EnumerationLoop(ParameterExpression enumerator, Expression loopContent)
    {
        var loop = While(
            Expression.Call(enumerator, typeof(IEnumerator).GetMethod("MoveNext")),
            loopContent);

        var enumeratorType = enumerator.Type;
        if (typeof(IDisposable).IsAssignableFrom(enumeratorType))
            return Using(enumerator, loop);

        if (!enumeratorType.IsValueType)
        {
            var disposable = Expression.Variable(typeof(IDisposable), "disposable");
            return Expression.TryFinally(
                loop,
                Expression.Block(new[] { disposable },
                    Expression.Assign(disposable, Expression.TypeAs(enumerator, typeof(IDisposable))),
                    Expression.IfThen(
                        Expression.NotEqual(disposable, Expression.Constant(null)),
                        Expression.Call(disposable, typeof(IDisposable).GetMethod("Dispose")))));
        }

        return loop;
    }

    public static Expression Using(ParameterExpression variable, Expression content)
    {
        var variableType = variable.Type;

        if (!typeof(IDisposable).IsAssignableFrom(variableType))
            throw new Exception($"'{variableType.FullName}': type used in a using statement must be implicitly convertible to 'System.IDisposable'");

        var getMethod = typeof(IDisposable).GetMethod("Dispose");

        if (variableType.IsValueType)
        {
            return Expression.TryFinally(
                content,
                Expression.Call(Expression.Convert(variable, typeof(IDisposable)), getMethod));
        }

        if (variableType.IsInterface)
        {
            return Expression.TryFinally(
                content,
                Expression.IfThen(
                    Expression.NotEqual(variable, Expression.Constant(null)),
                    Expression.Call(variable, getMethod)));
        }

        return Expression.TryFinally(
            content,
            Expression.IfThen(
                Expression.NotEqual(variable, Expression.Constant(null)),
                Expression.Call(Expression.Convert(variable, typeof(IDisposable)), getMethod)));
    }

    public static Expression While(Expression loopCondition, Expression loopContent)
    {
        var breakLabel = Expression.Label();
        return Expression.Loop(
            Expression.IfThenElse(
                loopCondition,
                loopContent,
                Expression.Break(breakLabel)),
            breakLabel);
    }
}

The ForEach without loopVar is useful to enumerate without getting the items. That's the case of Count() implementation.

EDIT: An updated and tested version is available in the NetFabric.Reflection NuGet package. Check its repository for the source code.

Answer 2

I got lost somewhere in the middle of your question (and if I've interpreted it incorrectly, please tell me, and I'll dive back into it), but I think this is what you're after:

public static Expression ForEach(Expression collection, ParameterExpression loopVar, Expression loopContent)
{
    var elementType = loopVar.Type;
    var enumerableType = typeof(IEnumerable<>).MakeGenericType(elementType);
    var enumeratorType = typeof(IEnumerator<>).MakeGenericType(elementType);

    var enumeratorVar = Expression.Variable(enumeratorType, "enumerator");
    var getEnumeratorCall = Expression.Call(collection, enumerableType.GetMethod("GetEnumerator"));
    var enumeratorAssign = Expression.Assign(enumeratorVar, getEnumeratorCall);

    // The MoveNext method's actually on IEnumerator, not IEnumerator<T>
    var moveNextCall = Expression.Call(enumeratorVar, typeof(IEnumerator).GetMethod("MoveNext"));

    var breakLabel = Expression.Label("LoopBreak");

    var loop = Expression.Block(new[] { enumeratorVar },
        enumeratorAssign,
        Expression.Loop(
            Expression.IfThenElse(
                Expression.Equal(moveNextCall, Expression.Constant(true)),
                Expression.Block(new[] { loopVar },
                    Expression.Assign(loopVar, Expression.Property(enumeratorVar, "Current")),
                    loopContent
                ),
                Expression.Break(breakLabel)
            ),
        breakLabel)
    );

    return loop;
}

To use it, you need to supply a collection to iterate over, an expression to substitute into the body of the loop, and a ParameterExpression which is used by the loop body expression, which will be assigned to the loop variable on each loop iteration.

I think sometimes examples speak louder than words...

var collection = Expression.Parameter(typeof(List<string>), "collection");
var loopVar = Expression.Parameter(typeof(string), "loopVar");
var loopBody = Expression.Call(typeof(Console).GetMethod("WriteLine", new[] { typeof(string) }), loopVar);
var loop = ForEach(collection, loopVar, loopBody);
var compiled = Expression.Lambda<Action<List<string>>>(loop, collection).Compile();
compiled(new List<string>() { "a", "b", "c" });

EDIT: As Jeroem Mostert correctly points out in the comments, this doesn't quite mirror the "real" behaviour of a foreach loop: this would make sure that it disposes the enumerator. (It would also create a new instance of the loop variable for each iteration, but that doesn't make sense with expressions). Implementing this is just a matter of turning the handle if you feel motivated enough!

---

For anyone watching at home, I've got a similar method for generating 'for' loops:

public static Expression For(ParameterExpression loopVar, Expression initValue, Expression condition, Expression increment, Expression loopContent)
{
    var initAssign = Expression.Assign(loopVar, initValue);

    var breakLabel = Expression.Label("LoopBreak");

    var loop = Expression.Block(new[] { loopVar },
        initAssign,
        Expression.Loop(
            Expression.IfThenElse(
                condition,
                Expression.Block(
                    loopContent,
                    increment
                ),
                Expression.Break(breakLabel)
            ),
        breakLabel)
    );

    return loop;
}

This is equivalent to the following statement, where the pseudo-variables match the Expressions in the method above:

for (loopVar = initValue; condition; increment)
{
    loopContent
}

Again, loopContent, condition, and increment are Expressions which uses loopVar, and loopVar is assigned on every iteration.