In Python: How to remove an object from a list if it is only referenced in that list?

Question

I want to keep track of objects of a certain type that are currently in use. For example: Keep track of all instances of a class or all classes that have been created by a metaclass.

It is easy to keep track of instances like this:

class A():
    instances = []
    def __init__(self):
        self.instances.append(self)

But if an instance is not referenced anywhere outside of that list it will not be needed anymore and I do not want to process that instance in a potentially time consuming loop.

I tried to remove objects that are only referenced in the list using sys.getrefcount.

for i in A.instances:
    if sys.getrefcount(i) <=3: # in the list, in the loop and in getrefcount
        # collect and remove after the loop

The problem I have is that the reference count is very obscure. Opening a new shell and creating a dummy class with no content returns 5 for

sys.getrefcount(DummyClass)

Another idea is to copy the objects then deleting the list and checking which objects have been scheduled for garbage collecting and in the last step removing those objects. Something like:

Copy = copy(A.instances)
del A.instances
A.instances = [i for i in Copy if not copy_of_i_is_in_GC(i)]

The objects don't have to be removed immediately when the reference count goes to 0. I just don't want to waste too much ressources on objects that are not used anymore.

Answer 1

This answer is the same as Kevin's but I was working up an example implementation with weak references and am posting it here. Using weak references solves the problem where an object is referenced by the self.instance list, so it will never be deleted.

One of the things about creating a weak reference for an object is that you can include a callback when the object is deleted. There are issues such as the callback not happening when the program exits... but that may be what you want anyway.

import threading
import weakref

class A(object):
    instances = []
    lock = threading.RLock()

    @classmethod
    def _cleanup_ref(cls, ref):
        print('cleanup') # debug
        with cls.lock:
            try:
                cls.instances.remove(ref)
            except ValueError:
                pass

    def __init__(self):
        with self.lock:
            self.instances.append(weakref.ref(self, self._cleanup_ref))

# test
test = [A() for _ in range(3)]
for i in range(3,-1,-1):
    assert len(A.instances) == i
    if test:
        test.pop()

print("see if 3 are removed at exit")
test = [A() for _ in range(3)]

Answer 2

The standard way to solve this problem is through weak references. The basic idea is that you keep a list of weak references to objects instead of the objects themselves, and periodically prune the dead weak references from the list.

For dictionaries and sets, there are some more abstract types such as weakref.WeakKeyDictionary() which can be used when you want to put weak references in more complex places like the keys of a dictionary. These types do not require manual pruning.