Why set a bound method to python object create a circular reference?

Question

I'm working in Python 2.7 and I fond that issue that puzzling me.

That is the simplest example:

>>> class A(object):
    def __del__(self):
        print("DEL")
    def a(self):
        pass

>>> a = A()
>>> del a
DEL

That is OK like expected... now I'm trying to change the a() method of object a and what happen is that after change it I can't delete a any more:

>>> a = A()
>>> a.a = a.a
>>> del a

Just to do some checks I've print the a.a reference before and after the assignment

>>> a = A()
>>> print a.a
<bound method A.a of <__main__.A object at 0xe86110>>
>>> a.a = a.a
>>> print a.a
<bound method A.a of <__main__.A object at 0xe86110>>

Finally I used objgraph module to try to understand why the object is not released:

>>> b = A()
>>> import objgraph
>>> objgraph.show_backrefs([b], filename='pre-backref-graph.png')

>>> b.a = b.a
>>> objgraph.show_backrefs([b], filename='post-backref-graph.png')

As you can see in the post-backref-graph.png image there is a __self__ references in b that have no sense for me because the self references of instance method should be ignored (as was before the assignment).

Somebody can explain why that behaviour and how can I work around it?

Answer 1

When you write a.a, it effectively runs:

A.a.__get__(a, A)

because you are not accessing a pre-bound method but the class' method that is being bound at runtime.

When you do

a.a = a.a

you effectively "cache" the act of binding the method. As the bound method has a reference to the object (obviously, as it has to pass self to the function) this creates a circular reference.

---

So I'm modelling your problem like:

class A(object):
    def __del__(self):
        print("DEL")
    def a(self):
        pass

def log_all_calls(function):
    def inner(*args, **kwargs):
        print("Calling {}".format(function))

        try:
            return function(*args, **kwargs)
        finally:
            print("Called {}".format(function))

    return inner

a = A()
a.a = log_all_calls(a.a)

a.a()

You can use weak references to bind on demand inside log_all_calls like:

import weakref

class A(object):
    def __del__(self):
        print("DEL")
    def a(self):
        pass

def log_all_calls_weakmethod(method):
    cls = method.im_class
    func = method.im_func
    instance_ref = weakref.ref(method.im_self)
    del method

    def inner(*args, **kwargs):
        instance = instance_ref()

        if instance is None:
            raise ValueError("Cannot call weak decorator with dead instance")

        function = func.__get__(instance, cls)

        print("Calling {}".format(function))

        try:
            return function(*args, **kwargs)
        finally:
            print("Called {}".format(function))

    return inner

a = A()
a.a = log_all_calls_weakmethod(a.a)

a.a()

This is really ugly, so I would rather extract it out to make a weakmethod decorator:

import weakref

def weakmethod(method):
    cls = method.im_class
    func = method.im_func
    instance_ref = weakref.ref(method.im_self)
    del method

    def inner(*args, **kwargs):
        instance = instance_ref()

        if instance is None:
            raise ValueError("Cannot call weak method with dead instance")

        return func.__get__(instance, cls)(*args, **kwargs)

    return inner

class A(object):
    def __del__(self):
        print("DEL")
    def a(self):
        pass

def log_all_calls(function):
    def inner(*args, **kwargs):
        print("Calling {}".format(function))

        try:
            return function(*args, **kwargs)
        finally:
            print("Called {}".format(function))

    return inner

a = A()
a.a = log_all_calls(weakmethod(a.a))

a.a()

Done!

---

FWIW, not only does Python 3.4 not have these issues, it also has WeakMethod pre-built for you.