Can I remove items from a ConcurrentDictionary from within an enumeration loop of that dictionary?

Question

So for example:

ConcurrentDictionary<string,Payload> itemCache = GetItems();  foreach(KeyValuePair<string,Payload> kvPair in itemCache) {     if(TestItemExpiry(kvPair.Value))     {   // Remove expired item.         itemCache.TryRemove(kvPair.Key, out Payload removedItem);     } } 

Obviously with an ordinary Dictionary<K,V> this will throw an exception, because removing items changes the dictionary's internal state during the life of the enumeration. It's my understanding that this is not the case for a ConcurrentDictionary, as the provided IEnumerable handles internal state changing. Am I understanding this right? Is there a better pattern to use?

Answer 1

It's strange to me that you've now received two answers that seem to confirm you can't do this. I just tested it myself and it worked fine without throwing any exception.

Below is the code I used to test the behavior, followed by an excerpt of the output (around when I pressed 'C' to clear the dictionary in a foreach and S immediately afterwards to stop the background threads). Notice that I put a pretty substantial amount of stress on this ConcurrentDictionary: 16 threading timers each attempting to add an item roughly every 15 milliseconds.

It seems to me this class is quite robust, and worth your attention if you're working in a multithreaded scenario.

Code

using System; using System.Collections.Concurrent; using System.Collections.Generic; using System.Threading;  namespace ConcurrencySandbox {     class Program {         private const int NumConcurrentThreads = 16;         private const int TimerInterval = 15;          private static ConcurrentDictionary<int, int> _dictionary;         private static WaitHandle[] _timerReadyEvents;         private static Timer[] _timers;         private static volatile bool _timersRunning;          [ThreadStatic()]         private static Random _random;         private static Random GetRandom() {             return _random ?? (_random = new Random());         }          static Program() {             _dictionary = new ConcurrentDictionary<int, int>();             _timerReadyEvents = new WaitHandle[NumConcurrentThreads];             _timers = new Timer[NumConcurrentThreads];              for (int i = 0; i < _timerReadyEvents.Length; ++i)                 _timerReadyEvents[i] = new ManualResetEvent(true);              for (int i = 0; i < _timers.Length; ++i)                 _timers[i] = new Timer(RunTimer, _timerReadyEvents[i], Timeout.Infinite, Timeout.Infinite);              _timersRunning = false;         }          static void Main(string[] args) {             Console.Write("Press Enter to begin. Then press S to start/stop the timers, C to clear the dictionary, or Esc to quit.");             Console.ReadLine();              StartTimers();              ConsoleKey keyPressed;             do {                 keyPressed = Console.ReadKey().Key;                 switch (keyPressed) {                     case ConsoleKey.S:                         if (_timersRunning)                             StopTimers(false);                         else                             StartTimers();                          break;                     case ConsoleKey.C:                         Console.WriteLine("COUNT: {0}", _dictionary.Count);                         foreach (var entry in _dictionary) {                             int removedValue;                             bool removed = _dictionary.TryRemove(entry.Key, out removedValue);                         }                         Console.WriteLine("COUNT: {0}", _dictionary.Count);                          break;                 }              } while (keyPressed != ConsoleKey.Escape);              StopTimers(true);         }          static void StartTimers() {             foreach (var timer in _timers)                 timer.Change(0, TimerInterval);              _timersRunning = true;         }          static void StopTimers(bool waitForCompletion) {             foreach (var timer in _timers)                 timer.Change(Timeout.Infinite, Timeout.Infinite);              if (waitForCompletion) {                 WaitHandle.WaitAll(_timerReadyEvents);             }              _timersRunning = false;         }          static void RunTimer(object state) {             var readyEvent = state as ManualResetEvent;             if (readyEvent == null)                 return;              try {                 readyEvent.Reset();                  var r = GetRandom();                 var entry = new KeyValuePair<int, int>(r.Next(), r.Next());                 if (_dictionary.TryAdd(entry.Key, entry.Value))                     Console.WriteLine("Added entry: {0} - {1}", entry.Key, entry.Value);                 else                     Console.WriteLine("Unable to add entry: {0}", entry.Key);              } finally {                 readyEvent.Set();             }         }     } } 

Output (excerpt)

cAdded entry: 108011126 - 154069760   // <- pressed 'C' Added entry: 245485808 - 1120608841 Added entry: 1285316085 - 656282422 Added entry: 1187997037 - 2096690006 Added entry: 1919684529 - 1012768429 Added entry: 1542690647 - 596573150 Added entry: 826218346 - 1115470462 Added entry: 1761075038 - 1913145460 Added entry: 457562817 - 669092760 COUNT: 2232                           // <- foreach loop begins COUNT: 0                              // <- foreach loop ends Added entry: 205679371 - 1891358222 Added entry: 32206560 - 306601210 Added entry: 1900476106 - 675997119 Added entry: 847548291 - 1875566386 Added entry: 808794556 - 1247784736 Added entry: 808272028 - 415012846 Added entry: 327837520 - 1373245916 Added entry: 1992836845 - 529422959 Added entry: 326453626 - 1243945958 Added entry: 1940746309 - 1892917475 

Also note that, based on the console output, it looks like the foreach loop locked out the other threads that were trying to add values to the dictionary. (I could be wrong, but otherwise I would've guessed you would've seen a bunch of "Added entry" lines between the "COUNT" lines.)

Answer 2

Just to confirm that the official documentation explicitly states that it is safe:

The enumerator returned from the dictionary is safe to use concurrently with reads and writes to the dictionary, however it does not represent a moment-in-time snapshot of the dictionary. The contents exposed through the enumerator may contain modifications made to the dictionary after GetEnumerator was called.