How many times can `__del__` be called per object in Python?

Question

I saw some code where __del__ was being called explicitly on an object which I found curious, so I tried to play with it a bit to understand how it works.

I tried the following code (I understand using __del__ in and of itself is potentially bad, but I'm just trying to edify myself here):

class A:
    def __init__(self, b):
        self.a = 123
        self.b = b
        print "a is {}".format(self.a)
    def __del__(self):
        self.a = -1
        print "a is {}".format(self.a)
        self.b.dont_gc_me = self
    def foo(self):
        self.a = 9999999
        print "a is {}".format(self.a)

class Foo:
    def __init__(self):
        self.a = 800
    def __del__(self):
        self.a = 0

and then tried (in the IPython console) the following:

In [92]: f = Foo()

In [93]: a = A(f)
a is 123

In [94]: a = None
a is -1

In [95]: f.__dict__
Out[95]: {'a': 800, 'dont_gc_me': <__main__.A instance at 0x2456830>}

In [96]: f = None

In [97]: 

I see that __del__ is only called once even though the instance a of A has been kept alive by the instance f of Foo, and when I set the latter to None, I don't see the destructor being called the second time.

The Python docs here say:

Note that it is possible (though not recommended!) for the __del__() method to postpone destruction of the instance by creating a new reference to it. It may then be called at a later time when this new reference is deleted. It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits.

This seems to imply that the __del__method may be called again, though there is no guarantee it will. So my question is: is there some circumstance where __del__ will be called again? (I thought setting f to None above would do it, but it didn't). Any other nuances worth noting?

Answer 1

Here's a way:

xs = []
n = 0

class A:
    def __del__(self):
        global n
        n += 1
        print("time {} calling __del__".format(n))
        xs.append(self)

print("creating an A immediately thrown away")
A()
for _ in range(5):
    print("popping from xs")
    xs.pop()

That prints:

creating an A immediately thrown away
time 1 calling __del__
popping from xs
time 2 calling __del__
popping from xs
time 3 calling __del__
popping from xs
time 4 calling __del__
popping from xs
time 5 calling __del__
popping from xs
time 6 calling __del__

So, in short, there's no limit on how many times __del__ can be called. But don't rely on this - the language may change "the rules" here eventually.

Cycles complicate the situation because when a cycle is entirely trash, there's no predictable order in which the objects belonging to the cycle will get destroyed. Since it is a cycle, every object is reachable from every other object in the cycle, so executing a __del__ for one of the objects in the cycle may reference an object that's already been destroyed. Major headaches there, so CPython simply refuses to collect a cycle at least one of whose objects has a __del__ method.

However, it's no problem if an object "hanging off" a trash cycle has a __del__ method, provided the latter object is not itself in a trash cycle. Example:

class A:
    def __del__(self):
        print("A is going away")

class C:
    def __init__(self):
        self.self = self
        self.a = A()

Then:

>>> c = C()
>>> import gc
>>> gc.collect()  # nothing happens
0
>>> c = None  # now c is in a trash self-cycle
>>> gc.collect()  # c.a.__del__ *is* called
A is going away
2

So that's the moral of this story: if you have an object that "needs" to run a destructor, but may be in a cycle, put the __del__ in a simple object referenced by the original object. Like so:

class CriticalResource:
    def __init__(self, resource):
        self.r = resource

    def __del__(self):
        self.r.close_nicely()  # whatever

class FancyObject:
    def __init__(self):
        # ...
        self.r = CriticalResource(get_some_resource())
        # ...

Now a FancyObject can be in as many cycles as you like. When it becomes trash, the cycles won't prevent CriticalResource's __del__ from being invoked.

Starting in Python 3.4

As @delnan noted in a comment, PEP 442 changes the CPython rules starting in Python 3.4 (but this will not be backported to any previous CPython release). __del__ methods will be executed at most once then (by gc - of course users can explicitly call them any number of times), and it will no longer matter whether an object with a __del__ method is part of cyclic trash.

The implementation will run all finalizers on all objects found in cyclic trash, and set a bit on each such object recording that its finalizer has been run. It does this before any objects are torn down, so no finalizer can access an object in an insane (partially or wholly destructed) state.

The downside for the implementation is that finalizers can change the object graph in arbitrary ways, so the current cyclic garbage collection ("gc") run has to give up if anything in the cyclic trash has become reachable again ("resurrected"). That's why finalizers are (will be) allowed to run at most once: else gc could be provoked into never making progress by a finalizer that resurrected dead objects in cyclic trash.

In effect, starting in 3.4, CPython's treatment of __del__ methods will be a lot like Java's treatment of finalizers.