How can OrderedDict know about the element order of an already instantiated dict?

Question

I was playing around with the OrderedDict type in Python 3.6 and was surprised by its behaviour. When I create a simple dict like this in IPython:

d = dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])

I get:

{'guido': 4127, 'jack': 4098, 'sape': 4139}

as an output, which doesn't preserve the order of elements at instantiation for some reason. Now, when I create an OrderedDict from d like this:

od = OrderedDict(d)

the output is:

OrderedDict([('sape', 4139), ('guido', 4127), ('jack', 4098)])

Now I ask myself, how can the OrderedDict-constructor know about the order of elements at instantiation of d? And does it always behave the same, such that I can rely on the order of elements in the OrderedDict?

I was already reading the Python docs about dictionaries and OrderedDicts but I didn't find an answer to my question.

The output from (sys.version):

In[22]: sys.version
Out[22]: '3.6.1 (default, Apr  4 2017, 09:40:21) \n[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)]'

Answer 1

In 3.6, as an implementation detail, all dicts are ordered. You're being fooled by IPython: Before 3.6, the order of keys was arbitrary, so for user-friendliness, IPython's interactive output for dict and set (where normal Python would just print the repr) sorts the keys. That's why your dict appears to be in alphabetical order. It's possible IPython might eventually drop that behavior when running on 3.6+, since as you've noticed, it is quite confusing.

If you explicitly print, rather than relying on ipython to output the results of the previous expression for you, you'll bypass ipython's REPL magic and see the "natural" order. Same goes for just about any other means of interacting with the dict, since iteration will proceed in the expected order.

Answer 2

It's now obvious that the custom hook (sys.displayhook) that IPython uses to display output is pretty printing things (using it's own pretty printer).

By directly calling displayhook you can see how it ruins the insertion order:

In [1]: from sys import displayhook
   ...: displayhook({'1': 0, '0': 1})
Out[1]: {'0': 1, '1': 0}

In addition, if you grabbed the dictionary str instead (sending a string to be displayed instead of a dict object) you'd get the correct and expected order:

In [2]: d = dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
   ...: d
Out[2]: {'guido': 4127, 'jack': 4098, 'sape': 4139}

In [3]: str(dict(t))
Out[3]: "{'sape': 4139, 'guido': 4127, 'jack': 4098}"

similarly by printing it.

I'm not sure why IPython does this with 3.6, it was quite confusing (edit: see relevant issue on GitHub). In your standard Python REPL, this behavior won't manifest since sys.displayhook isn't implemented to do any pretty printing.

---

The dict d you've created does maintain insertion order, that's why the OrderedDict is maintaining that same order.

The fact that it does is, of course, an implementation detail. Until that is changed (and it does appear that it will) you should stick to using OrderedDict to reliably maintain order across implementations.

---

By the way, if you want this disabled, you could start IPython with the --no-pprint option which disables its pretty printer:

➜ ipython --no-banner --no-pprint 

In [1]: dict([('sape', 4139), ('guido', 4127), ('jack', 4098)])
Out[1]: {'sape': 4139, 'guido': 4127, 'jack': 4098}