Method calls inside LINQ select (efficiency)

Question

I have an efficiency-based question regarding LINQ vs a standard foreach loop.

I have a method, which does approximately the following

var foos = users.Select(this.GenerateNullableFoo).Where(foo => foo != null);

This turned out, for some reason, to be horrendously slow. Using timestamps I was able to determine that the point of slowness was exactly at the iteration of the select (time from final return statement to beginning of next method call) was vastly slower (13 seconds) than anything else (total time <= 14 seconds).

When replaced with the following, the entire process time was reduced to under 1 second per iteration.

New code:

var foos= new List<Foo>();

foreach (var user in users)
{
    var foo = GenerateNullableFoo(user);

        foo.IfNotNull(f => foos.Add(foo));
}

As far as I'm aware, my code works and there is nothing wrong with the new code, however, I am completely baffled as to why using the Select above was more than 10 times slower than a foreach doing seemingly the same process. Is there any insight as to what differences might account for this?

Also, the entire List is not saved until after said select/foreach at which time a batch call is made to the db.

EDIT: The entire code block from the point of origin to finish (and termination) is as follows:

//Code snippet pictured above
_repo.SaveFoo(Foos);
//at this point, the code terminates and is finished

Answer 1

As others have said, a LINQ expression is not projected to a result set until it is iterated on by code which means all you have done is declare the logic.

The code you have not posted is then causing the LINQ expression to be projected, and most likely many times.

The shortest way to solve this is simply add .ToList() to your expression, this will project the expression immediately meaning other code can use the in-memory and already projected list without causing additional projections/calculations/etc etc..

Cheers, Aaron