ToList method in Linq

Question

If I am not wrong, the ToList() method iterate on each element of provided collection and add them to new instance of List and return this instance.Suppose an example

//using linq
list = Students.Where(s => s.Name == "ABC").ToList();

//traditional way
foreach (var student in Students)
{
  if (student.Name == "ABC")
    list.Add(student);
}

I think the traditional way is faster, as it loops only once, where as of above of Linq iterates twice once for Where method and then for ToList() method.

The project I am working on now has extensive use of Lists all over and I see there is alot of such kind of use of ToList() and other Methods that can be made better like above if I take list variable as IEnumerable and remove .ToList() and use it further as IEnumerable.

Do these things make any impact on performance?

Answer 1

Do these things make any impact on performance?

That depends on your code. Most of the time, using LINQ does cause a small performance hit. In some cases, this hit can be significant for you, but you should avoid LINQ only when you know that it is too slow for you (i.e. if profiling your code showed that LINQ is reason why your code is slow).

But you're right that using ToList() too often can cause significant performance problems. You should call ToList() only when you have to. Be aware that there are also cases where adding ToList() can improve performance a lot (e.g. when the collection is loaded from database every time it's iterated).

Regarding the number of iterations: it depends on what exactly do you mean by “iterates twice”. If you count the number of times MoveNext() is called on some collection, then yes, using Where() this way leads to iterating twice. The sequence of operations goes like this (to simplify, I'm going to assume that all items match the condition):

1. Where() is called, no iteration for now, Where() returns a special enumerable.
2. ToList() is called, calling MoveNext() on the enumerable returned from Where().
3. Where() now calls MoveNext() on the original collection and gets the value.
4. Where() calls your predicate, which returns true.
5. MoveNext() called from ToList() returns, ToList() gets the value and adds it to the list.
6. …

What this means is that if all n items in the original collection match the condition, MoveNext() will be called 2n times, n times from Where() and n times from ToList().

Answer 2

var list = Students.Where(s=>s.Name == "ABC"); 

This will only create a query and not loop the elements until the query is used. By calling ToList() will first then execute the query and thus only loop your elements once.

List<Student> studentList = new List<Student>();
var list = Students.Where(s=>s.Name == "ABC");
foreach(Student s in list)
{
    studentList.add(s);
}

this example will also only iterate once. Because its only used once. Keep in mind that list will iterate all students everytime its called.. Not only just those whose names are ABC. Since its a query.

And for the later discussion Ive made a testexample. Perhaps its not the very best implementation of IEnumable but it does what its supposed to do.

First we have our list

public class TestList<T> : IEnumerable<T>
{
    private TestEnumerator<T> _Enumerator;

    public TestList()
    {
        _Enumerator = new TestEnumerator<T>();
    }

    public IEnumerator<T> GetEnumerator()
    {
        return _Enumerator;
    }

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
    {
        throw new NotImplementedException();
    }

    internal void Add(T p)
    {
        _Enumerator.Add(p);
    }
}

And since we want to count how many times MoveNext is called we have to implement our custom enumerator aswel. Observe in MoveNext we have a counter that is static in our program.

public class TestEnumerator : IEnumerator { public Item FirstItem = null; public Item CurrentItem = null;

    public TestEnumerator()
    {
    }

    public T Current
    {
        get { return CurrentItem.Value; }
    }

    public void Dispose()
    {

    }

    object System.Collections.IEnumerator.Current
    {
        get { throw new NotImplementedException(); }
    }

    public bool MoveNext()
    {
        Program.Counter++;
        if (CurrentItem == null)
        {
            CurrentItem = FirstItem;
            return true;
        }
        if (CurrentItem != null && CurrentItem.NextItem != null)
        {
            CurrentItem = CurrentItem.NextItem;
            return true;
        }
        return false;
    }

    public void Reset()
    {
        CurrentItem = null;
    }

    internal void Add(T p)
    {
        if (FirstItem == null)
        {
            FirstItem = new Item<T>(p);
            return;
        }
        Item<T> lastItem = FirstItem;
        while (lastItem.NextItem != null)
        {
            lastItem = lastItem.NextItem;
        }
        lastItem.NextItem = new Item<T>(p);
    }
}

And then we have a custom item that just wraps our value

public class Item<T>
{
    public Item(T item)
    {
        Value = item;
    }

    public T Value;

    public Item<T> NextItem;
}

To use the actual code we create a "list" with 3 entries.

    public static int Counter = 0;
    static void Main(string[] args)
    {
        TestList<int> list = new TestList<int>();
        list.Add(1);
        list.Add(2);
        list.Add(3);

        var v = list.Where(c => c == 2).ToList(); //will use movenext 4 times
        var v = list.Where(c => true).ToList();   //will also use movenext 4 times


        List<int> tmpList = new List<int>(); //And the loop in OP question
        foreach(var i in list)
        {
            tmpList.Add(i);
        }                                    //Also 4 times.
    }

And conclusion? How does it hit performance? The MoveNext is called n+1 times in this case. Regardless of how many items we have. And also the WhereClause does not matter, he will still run MoveNext 4 times. Because we always run our query on our initial list. The only performance hit we will take is the actual LINQ framework and its calls. The actual loops made will be the same.

And before anyone asks why its N+1 times and not N times. Its because he returns false the last time when he is out of elements. Making it the number of elements + end of list.