C# ConcurrentBag memory consumption when iterating

Question

It's easier if I start by posting the code:

static void Main(string[] args)
{
    List<double> testLst = new List<double>();
    for (int i = 0; i < 20000000; i++) { testLst.Add(i); }

I've populated a List with 20,000,000 elements. I see in the Task Manager that the process is using ~300MB. If I iterate through the list using a foreach loop:

    foreach (var a in testLst.Take(10)) 
    {
        Console.WriteLine(a);
    }
}

memory usage does not increase (I've put a breakpoint on Console.WriteLine and, as I said, I'm measuring it using the Task Manager). Now, if I replace the List with a ConcurrentBag:

static void Main(string[] args)
{
    ConcurrentBag<double> testCB = new ConcurrentBag<double>();
    for (int i = 0; i < 20000000; i++) { testCB.Add(i); }

    foreach (var a in testCB.Take(10)) 
    {
        Console.WriteLine(a);
    }
}

memory usage is 450~500MB before the foreach-loop. The question is: why if inside the foreach-loop usage jumps to ~900MB?

I expect the ConcurrentBag to consume more memory compared to the List, but I don't understand why so much memory is being used for the iteration.

(I'm using the ConcurrentBag in a similar but different scenario, I know that in this case it would not make sense to use it)

Answer 1

From the ConcurrentBag.GetEnumerator docs (emphasis mine):

The enumeration represents a moment-in-time snapshot of the contents of the bag. It does not reflect any updates to the collection after GetEnumerator was called. The enumerator is safe to use concurrently with reads from and writes to the bag.

Looking at the source, you can see it creates a copy of the bag:

public IEnumerator<T> GetEnumerator()
{
    // Short path if the bag is empty
    if (m_headList == null)
        return new List<T>().GetEnumerator(); // empty list

    bool lockTaken = false;
    try
    {
        FreezeBag(ref lockTaken);
        return ToList().GetEnumerator();
    }
    finally
    {
        UnfreezeBag(lockTaken);
    }
}

Like its name implies, ToList() returns a List<T> (it's not the extension method, it's a private member function).

As a side note, that return new List<T>().GetEnumerator(); line isn't pretty... that could have written return Enumerable.Empty<T>().GetEnumerator(); instead.