C# Why does Except and Where Enumerable Give This Weird Result?

Question

I have been debugging this production bug for sometime and I'm desperate for help and also it's interesting to me.

I have simplified the code logic and added some print-out for debugging:

int[] a = { 2,2,2 };
var b = a.Where(x => x==2);

for(int i = 0; i < 3; i++)
{
    var c = b.Where(x => x==i);

    Console.WriteLine("iter {0} before - B Count: {1}, C Count: {2}", i, b.Count(), c.Count());

    if (c.Count() != b.Count())
        b = b.Except(c);

    Console.WriteLine("iter {0} after - B Count: {1}, C Count: {2}", i, b.Count(), c.Count());
}

Console.WriteLine("After Loop: B Count: {0}", b.Count());

Interestingly (Weirdly), the output is:

iter 0 before - B Count: 3, C Count: 0
iter 0 after - B Count: 1, C Count: 0
iter 1 before - B Count: 1, C Count: 0
iter 1 after - B Count: 1, C Count: 0
iter 2 before - B Count: 0, C Count: 0
iter 2 after - B Count: 0, C Count: 0
After Loop: B Count: 1

Question 1:

Why b.Count() == 0 at "iter 2 before". The only thing happens between "iter 1 after and iter 2 before" is

var c = b.Where(x => x==i);

Why does this code change b at all?

Question 2:

Why does b.Count() become back to 1 after the loop is finished?

I really appreciate everyone's help to guide me through this problem, thanks!

Answer 1

Due to linq's lazy evaluation, each of the Where and Except calls is executed every time you make a call to Count(). What's important to note here is that the value of i is not remembered, instead it takes the current value of i for all the previous Where calls.

Why b.Count() == 0 at "iter 2 before"

At this point, i has already been incremented to 2. When you call count, the entire linq query is evaluated using this value, meaning that the previously added 'Except` call will already remove everything!

Why does b.Count() become back to 1 after the loop is finished?

The loop ends when i equals 3. Now when you evaluate the entire linq query, none of the .Where(x => x==i) calls will return anything, which in turn means that none of the Except calls will remove anything (other than the duplicates).

Answer 2

For 1: "Why does this code change b at all?"

It doesn't. Add (at the start of the loop, before c):

Console.WriteLine("iter {0} before anything - B Count: {1}", i, b.Count());

b is unchanged.

For 2: that's because of the captured i. Change to:

for(int tmp = 0 ; tmp < 3 ; tmp++) {
      int i = tmp;
    // ...
}

and try again. Now the i is scoped inside the loop, and is captured accordingly (the iterator variable in a for loop is scoped outside the loop, for complicated reasons; in a foreach loop, whether the iterator variable is scoped inside or outside the loop depends on the version of C# you are using)

Answer 3

Except acts like its SQL counterpart in that it eliminates duplicates. Because of this, it behaves as if it called Distinct on your list.

So when you first execute Except

if (c.Count() != b.Count()) // c.Count() == 0, b.Count() == 3
    b = b.Except(c);

You only get the unique elements of b, of which there is only one.