Can one iterate a dictionary being modified by another thread without raising exceptions?

Question

I have a dictionary being updated by one thread, and in another thread I'd like to iterate over its values. Normally I'd use a lock, but this code is very performance-critical, and I want to avoid that if at all possible.

A special feature of my case is that I don't care about absolute correctness of the iterator; if it misses entries that were removed after iteration started, or picks up ones added afterwards, that's fine. I only require that it doesn't raise any sort of 'dictionary size changed during iteration' exception.

Given this relaxed constraint on correctness, is there an efficient way to iterate the dictionary without using a lock?

Note: I'm aware that keys() is threadsafe in Python 2.x, but since that behavior has changed in 3.x, I want to avoid it.

Answer 1

No personal experience with this, but I read this some time ago: http://www.python.org/dev/peps/pep-3106/

These operations are thread-safe only to the extent that using them in a thread-unsafe way may cause an exception but will not cause corruption of the internal representation.

As in Python 2.x, mutating a dict while iterating over it using an iterator has an undefined effect and will in most cases raise a RuntimeError exception. (This is similar to the guarantees made by the Java Collections Framework.)

Answer 2

I would consider using a lock long enough to retrieve the values that you want to iterate over:

with lock:
    values = the_dict.values() # python 2
    # values = list(the_dict.values()) # python 3
for value in values:
    # do stuff

Or, you could try it without a lock and catch RuntimeError, and if you get one, try to retrieve the values again.

[edit] Below slightly rephrased per J.F. Sebastian's suggesion:

while True:
    try:
        values = list(the_dict.values())
        break
    except RuntimeError:
        pass

I personally would go with the lock.