Why isn't Pickle calling __new__ like the documentation says?

Question

The documentation for Pickle specifically says:

Instances of a new-style class C are created using:

obj = C.__new__(C, *args)

Attempting to take advantage of this, I created a singleton with no instance attributes or methods:

class ZeroResultSentinel(object):
    instance = None
    def __new__(cls, *args):
        if not cls.instance:
            cls.instance = super(ZeroResultSentinel, cls).__new__(cls, *args)
        return cls.instance

(This class is used in a caching layer to differentiate a no-result result from nothing in the cache.)

The singleton works great (every call to ZeroResultSentinel() results in the same instance in memory, and ZeroResultSentinel() == ZeroResultSentinel() evaluates to True). And I can pickle and unpickle the instance without errors. However, when I unpickle it, I get a different instance. So I placed a breakpoint within __new__. I hit the breakpoint every time I call ZeroResultSentinel(), but I do not hit a breakpoint when I unpickle a pickled ZeroResultSentinel. This is in direct contradiction to the documentation. So am I doing something wrong, or is the documentation incorrect?

Answer 1

The documentation doesn't really make it clear, but your __new__ method will only be used for pickle protocol 2 and up:

>>> class Foo(object):
...     def __new__(cls):
...         print "New"
...         return object.__new__(cls)
...
>>> foo = Foo()
New
>>> pickle.loads(pickle.dumps(foo, protocol=0))
<__main__.Foo object at 0x00000000025E9A20>
>>> pickle.loads(pickle.dumps(foo, protocol=2))
New
<__main__.Foo object at 0x00000000022A3F60>

On Python 2, the default protocol is 0, so if you're using the default, you'll have to change that.