Python destructor called in the "wrong" order based on reference count

Question

As far as I understand Python destructors should be called when the reference count of an object reaches 0. But this assumption seems not to be correct. Look at the following code:

class A:
    def __init__(self, b):
        self.__b = b
        print("Construct A")
        
    def __del__(self):
        # It seems that the destructor of B is called here.
        print("Delete A")
        # But it should be called here
        
class B:
    def __init__(self):
        print("Construct B")
        
    def __del__(self):
        print("Delete B")
        
b = B()
a = A(b)

Outputs

Construct B                                                                                                                                                                                                                                   
Construct A                                                                                                                                                                                                                                   
Delete B                                                                                                                                                                                                                                      
Delete A

But A has a reference to B, so I would expect the following output:

Construct B                                                                                                                                                                                                                                   
Construct A                                                                                                                                                                                                                                   
Delete A                                                                                                                                                                                                                                      
Delete B

What am I not getting?

Answer 1

So, since the objects are still alive when the interpreter shuts down, you are actually not even guaranteed that __del__ will be called. At this point, the language makes no guarantees about when the finalizer is called.

From the docs:

It is not guaranteed that __del__() methods are called for objects that still exist when the interpreter exits.

Note, if you change the script to:

(py38) 173-11-109-137-SFBA:~ juan$ cat test.py
class A:
    def __init__(self, b):
        self.__b = b
        print("Construct A")

    def __del__(self):
        # It seems that the destructor of B is called here.
        print("Delete A")
        # But it should be called here

class B:
    def __init__(self):
        print("Construct B")

    def __del__(self):
        print("Delete B")
b = B()
a = A(b)

del a
del b

Then, executed:

(py38) 173-11-109-137-SFBA:~ juan$ python test.py
Construct B
Construct A
Delete A
Delete B

Although del does not delete objects, it deletes references, so it forces the reference count to reach 0 while the interpreter is still running, so the order is as you would expect.

Sometimes, __del__ won't be called at all. A common circumstance is file-objects created by

f = open('test.txt')

That have live references in the global scope. If not closed explicitly, it might not call __del__ and the file will not flush and you won't get anything written. Which is a great reason to use the file object as a context-manager...

Answer 2

Per the comments elsewhere on this question, you probably don't want to use __del__; it's not really a destructor in the C++ sense. You probably want to make the objects into context managers (by writing __enter__ and __exit__ methods) and use them in the with statement, and/or give them close methods which need to be called explicitly.

However, to answer the question as given: the reason is that both objects have references from the global variables a and b; neither reference count ever goes to zero. The destructor is called at the end when the python interpreter is shutting down and all the non-zero-count objects are being collected.

To see the behaviour you expect, put the a and b variables in a function so that the reference counts go to zero during the main part of execution.

class A:
    def __init__(self, b):
        self.__b = b
        print("Construct A")

    def __del__(self):
        print("Delete A")

class B:
    def __init__(self):
        print("Construct B")

    def __del__(self):
        print("Delete B")

def foo():
    b = B()
    a = A(b)

foo()

Answer 3

Among the things you're missing, there's a reference cycle. It goes roughly a->b->B->B.__init__->B.__init__.__globals__->a:

• Your A instance has a reference to its __dict__, which has a reference to your B instance.
• Your B instance has a reference to your B class.
• Your B class has a reference to its __dict__, which has references to all of B's methods. (Technically, if you try to access B.__dict__ yourself, you'll get a mappingproxy wrapping B's "real __dict__". B has a reference to the real dict, not the proxy.)
• B's methods each have a reference to their global variable dict.
• The global variable dict has a reference to your A instance (because this dict is where the a global variable is).

When reclaiming objects in a reference cycle, there are no guarantees as to what order __del__ methods are executed in.

If you don't believe the reference cycle exists, it's fairly straightforward to demonstrate the existence of these references:

import gc

print(a.__dict__ in gc.get_referents(a))
print(b in gc.get_referents(a.__dict__))
print(B in gc.get_referents(b))
# this bypasses the mappingproxy
# never use this to modify a class's dict - you'll cause memory corruption
real_dict = next(d for d in gc.get_referents(B) if isinstance(d, dict))
print(B.__init__ in gc.get_referents(real_dict))
print(B.__init__.__globals__ in gc.get_referents(B.__init__))
print(a in gc.get_referents(B.__init__.__globals__))

All of these prints print True.

---

Aside from that, there are a few relevant points other answers have already brought up. Your objects survive to interpreter shutdown, so there is no guarantee that __del__ will be called at all. Also, __del__ is a finalizer, not a destructor. It doesn't have anywhere near the same kind of guarantees that an actual destructor would have in a language like C++.