Memory leak when invoking __iadd__ via __get__ without using temporary

Question

trying modify a decorator not to use a weakref, I stumbled across the following behaviour:

import weakref

class descriptor(object):
    def __get__(self, instance, owner):
        return proxy(instance)

class proxy(object):
    def __init__(self, instance):
        self.instance = instance

    def __iadd__(self, other):
        return self

class A(object):
    descr = descriptor()

def is_leaky(test_fn):
    a = A()
    wr = weakref.ref(a)
    test_fn(a)
    del a
    return wr() is not None

def test1(a):
    tmp = a.descr
    tmp += object()

def test2(a):
    a.descr += object()

print(is_leaky(test1))  # gives False
print(is_leaky(test2))  # gives True!!!

This seems very odd to me, as I'd expect both cases to behave the same. Furthermore, from my understanding of reference counting and object lifetime, I was convinced that in both cases the object should be freed.

I have tested it both on python2.7 and python3.3.

Is this a bug or intentional behaviour? Is there a way to get both calls to have the expected results (free the object in question)?

I do not want to use a weakref in proxy because this destroys the correct object lifetime semantics for bound methods:

a = A()
descr = a.descr
del a   # a is kept alive since descr is a bound method to a
descr() # should execute a.descr() as expected

Answer 1

The two codepaths are not equivalent.

In-place add acts on two operators, the assignment target and the item added. In test1 that is temp, a local variable, and the in-place addition is translated to following:

temp = temp.__iadd__(object())

and since you return self and temp refers to the same object, this becomes temp = temp and that reference is cleaned up after the function exits.

In test2, you complicated matters, since now descriptors are getting involved again:

a.descr += object() 

becomes:

a.descr = A.__dict__['descr'].__get__(a, A).__iadd__(object())

so you assign the result of A.__dict__['descr'].__get__(a, A) to the instance attribute a.descr; the descriptor has no __set__() method and is not consulted.

But, and here is the catch, the proxy object holds a reference to a itself, a.descr.instance is a reference to a! You created a circular reference.

This reference keeps the object alive long enough to show through the weak reference, but as soon as the garbage collection process runs and breaks this cycle, a will be gone anyway.

Moral of this story? Don't use __iadd__ in combination with a non-data descriptor; include both __get__ and __set__ because you need to control what happens when the result is assigned back.