Does iterating over my_dict.keys() and modifying the values in the dictionary invalidate the iterator?

Question

My example is something like this

for my_key in my_dict.keys():
    my_dict[my_key].mutate()

Is the behavior of the above code defined (assuming my_dict is a dictionary and mutate is a method that mutates its object)? My concern is that mutating the values in the dictionary might invalidate the iterator over the keys of the dictionary.

On the other hand, the Python documentation of the keys method indicates that it returns a list. If so, I should (I think) be able to mutate the dictionary's values and still access every element of the dictionary safely as long as I don't mutate, add, or remove any of the keys. Is that correct?

Answer 1

Yes, that's safe, at least in all of the C-based implementations of Python. It's also safe to do:

for my_key in my_dict: # NOTE:  no .keys() here
    my_dict[my_key].mutate()

That is, the safety does not depend on materializing a list of the keys, and it's generally more efficient not to call .keys() just to iterate over the keys.

EDIT

Note that it's also fine to entirely replace values associated with existing keys during iteration:

for my_key in my_dict:
    my_dict[my_key] = something_new()

The docs could be clearer about this ;-) What isn't predictably safe is removing or adding keys during iteration. What you do to the values doesn't matter.

Answer 2

Yes. This isn't specific to Python, dictionaries, or even hash based data structures.

The internal structure of a dictionary is based entirely on its keys. If you change the keys around you can change the structure of the dictionary and if that happens in the middle of iterating directly over a dictionary the iteration behaviour becomes undefined. Specifically if you want to add or remove keys during iteration then you would use .keys() to avoid actually iterating over the dictionary to avoid this problem. If you mutate a key it becomes entirely broken, hence no mutable built in types are allowed as dictionary keys.

But the dictionary doesn't care about values. They have no impact on storage or structure whatsoever. They're just a side detail, a trivial part of the implementation. It especially doesn't care if you mutate the values - in fact it has no way of knowing that you did so.

This shouldn't be surprising. It's the keys that you iterate over. You store things by key and later retrieve them by key. You check if a key is in a dictionary, and if it isn't you might get a KeyError. The values are just these things that sit next to the keys, so that they can be found and returned when the key is found.

Similarly you can iterate over a list and mutate or replace its values while you do so, but don't append or delete anything because that affects the internal structure. The indices of a list are like its keys.

However you can't touch the values of a set because in a set, keys and values play the same role.