How can I tell when a LINQ query is enumerated?

Question

This might seem to be a silly question at first, but please read on.

I know that LINQ queries are deferred and only executed when the query is enumerated, but I'm having trouble figuring out exactly when that happens. Certainly in a For Each loop, the query would be enumerated. What's the rule of thumb to follow? I don't want to accidentally enumerate over my query twice if it's a huge result.

For example, does System.Linq.Enumerable.First enumerate over the whole query? I ask for performance reasons. I want to pass a LINQ result set to an ASP.NET MVC view, and I also want to pass the First element separately. Enumerating over the results twice would be painful.

It would be great to turn on some kind of flag that alerts me each time a LINQ query is enumerated. That way I could catch scenarios when I accidentally enumerate twice.

Answer 1

You can add your own logging quite easily to see what's going on. Other than that, the lazy/eager bit is reasonably clear. Basically it's lazy when it can be - any time the return type is IEnumerable<T> or IOrderedEnumerable<T>. It's possible for those to be lazy because you can't get at any of the data without calling GetEnumerator(). Compare that to First() for example - it has to return a value to you. It can't defer anything.

As a general point, if you want to make sure that a query won't be evaluated more than once, call ToList or ToArray on it, then use the results of that several times. Again, those methods have to return a list or an array immediately, neither of which allows for lazy population. The query is evaluated, but then it's effectively disconnected from the resulting populated collection - the query won't be executed again, however much you examine the list.

In addition to the lazy/eager question, there's streaming/non-streaming: will the method read everything from the source enumerable, or just "sip" at it, reading when it needs to. Again, in general LINQ will only read when it has to - so while Reverse is non-streaming (but still lazy), Where and Select are streaming.

Answer 2

There is no hard and fast rule as to when a LINQ query will be enumerated and when it won't. Partially because some methods will or won't based on the underlying type of the query source.

Here is a quick break down. This is not a complete break down by any means, mainly what I could come up with in 5 minutes.

Aggregate Functions

They enumerate the list entirely and immediately. They are usually spotted by the extension methods which return a scalar value. For example Sum, Min, Max, Count, Last etc ...

Note: Count and Last do not necessarily enumerate the entire list. If the underlying type is convertible to ICollection<T> they will instead use a more efficient method.

Front of the list Selectors

They only look at the first element of the list and potentially the second. They are First, FirstOrDefault, Single, SingleOrDefault.

The above is referencing the versions which do not take a predicate. If they take a predicate they are better classified as Inquiries (see below)

Inquiries

They will only enumerate the minimal amount of the list necessary to do the operation. This can be as little as 1 element and as many as the entire list.

Examples: Any, Contains

Create a new list and do no enumeration immediately.

This is the vast majority of the operators in LINQ. Their cost is incurred when the new list is enumerated. Examples: Select, Where, Group, Join, SkipWhile, Skip.