C# Lambda performance issues/possibilities/guidelines

Question

I'm testing performance differences using various lambda expression syntaxes. If I have a simple method:

public IEnumerable<Item> GetItems(int point) {     return this.items.Where(i => i.IsApplicableFor(point)); } 

then there's some variable lifting going on here related to point parameter because it's a free variable from lambda's perspective. If I would call this method a million times, would it be better to keep it as it is or change it in any way to improve its performance?

What options do I have and which ones are actually feasible? As I understand it is I have to get rid of free variables so compiler won't have to create closure class and instantiate it on every call to this method. This instantiation usually takes significant amount of time compared to non-closure versions.

The thing is I would like to come up with some sort of lambda writing guidelines that would generally work, because it seems I'm wasting some time every time I write a heavily hit lambda expression. I have to manually test it to make sure it will work, because I don't know what rules to follow.

Alternative method

& example console application code

I've also written a different version of the same method that doesn't need any variable lifting (at least I think it doesn't, but you guys who understand this let me know if that's the case):

public IEnumerable<Item> GetItems(int point) {     Func<int, Func<Item, bool>> buildPredicate = p => i => i.IsApplicableFor(p);     return this.items.Where(buildPredicate(point)); } 

Check out Gist here. Just create a console application and copy the whole code into Program.cs file inside namespace block. You will see that the second example is much much slower even though it doesn't use free variables.

A contradictory example

The reason why I would like to construct some lambda best usage guidelines is that I've met this problem before and to my surprise that one turned out to be working faster when a predicate builder lambda expression was used.

Now explain that then. I'm completely lost here because it may as well turn out I won't be using lambdas at all when I know I have some heavy use method in my code. But I would like to avoid such situation and get to the bottom of it all.

Edit

Your suggestions don't seem to work

I've tried implementing a custom lookup class that internally works similar to what compiler does with a free variable lambda. But instead of having a closure class I've implemented instance members that simulate a similar scenario. This is the code:

private int Point { get; set; } private bool IsItemValid(Item item) {     return item.IsApplicableFor(this.Point); }  public IEnumerable<TItem> GetItems(int point) {     this.Point = point;     return this.items.Where(this.IsItemValid); } 

Interestingly enough this works just as slow as the slow version. I don't know why, but it seems to do nothing else than the fast one. It reuses the same functionality because these additional members are part of the same object instance. Anyway. I'm now extremely confused!

I've updated Gist source with this latest addition, so you can test for yourself.

Answer 1

What makes you think that the second version doesn't require any variable lifting? You're defining the Func with a Lambda expression, and that's going to require the same bits of compiler trickery that the first version requires.

Furthermore, you're creating a Func that returns a Func, which bends my brain a little bit and will almost certainly require re-evaluation with each call.

I would suggest that you compile this in release mode and then use ILDASM to examine the generated IL. That should give you some insight into what code is generated.

Another test that you should run, which will give you more insight, is to make the predicate call a separate function that uses a variable at class scope. Something like:

private DateTime dayToCompare; private bool LocalIsDayWithinRange(TItem i) {     return i.IsDayWithinRange(dayToCompare); }  public override IEnumerable<TItem> GetDayData(DateTime day) {     dayToCompare = day;     return this.items.Where(i => LocalIsDayWithinRange(i)); } 

That will tell you if hoisting the day variable is actually costing you anything.

Yes, this requires more code and I wouldn't suggest that you use it. As you pointed out in your response to a previous answer that suggested something similar, this creates what amounts to a closure using local variables. The point is that either you or the compiler has to do something like this in order to make things work. Beyond writing the pure iterative solution, there is no magic you can perform that will prevent the compiler from having to do this.

My point here is that "creating the closure" in my case is a simple variable assignment. If this is significantly faster than your version with the Lambda expression, then you know that there is some inefficiency in the code that the compiler creates for the closure.

I'm not sure where you're getting your information about having to eliminate the free variables, and the cost of the closure. Can you give me some references?