Track changes to lists and dictionaries in python?

Question

I have a class where the instances of this class needs to track the changes to its attributes.

Example: obj.att = 2 would be something that's easily trackable by simply overriding the __setattr__ of obj.

However, there is a problem when the attribute I want to change is an object itself, like a list or a dict.

How would I be able to track things like obj.att.append(1) or obj.att.pop(2)?

I'm thinking of extending the list or the dictionary class, but monkey patching instances of those classes once the obj and the obj.att is both initialized so that obj gets notified when things like .append is called. Somehow, that doesn't feel very elegant.

The other way that I can think of would be passing an instance of obj into the list initialization, but that would break a lot of existing code plus it seems even less elegant than the previous method.

Any other ideas/suggestions? Is there a simple solution that I'm missing here?

Answer 1

I was curious how this might be accomplished when I saw the question, here is the solution I came up with. Not as simple as I would like it to be but it may be useful. First, here is the behavior:

class Tracker(object):
    def __init__(self):
        self.lst = trackable_type('lst', self, list)
        self.dct = trackable_type('dct', self, dict)
        self.revisions = {'lst': [], 'dct': []}


>>> obj = Tracker()            # create an instance of Tracker
>>> obj.lst.append(1)          # make some changes to list attribute
>>> obj.lst.extend([2, 3])
>>> obj.lst.pop()
3
>>> obj.dct['a'] = 5           # make some changes to dict attribute
>>> obj.dct.update({'b': 3})
>>> del obj.dct['a']
>>> obj.revisions              # check out revision history
{'lst': [[1], [1, 2, 3], [1, 2]], 'dct': [{'a': 5}, {'a': 5, 'b': 3}, {'b': 3}]}

Now the trackable_type() function that makes all of this possible:

def trackable_type(name, obj, base):
    def func_logger(func):
        def wrapped(self, *args, **kwargs):
            before = base(self)
            result = func(self, *args, **kwargs)
            after = base(self)
            if before != after:
                obj.revisions[name].append(after)
            return result
        return wrapped

    methods = (type(list.append), type(list.__setitem__))
    skip = set(['__iter__', '__len__', '__getattribute__'])
    class TrackableMeta(type):
        def __new__(cls, name, bases, dct):
            for attr in dir(base):
                if attr not in skip:
                    func = getattr(base, attr)
                    if isinstance(func, methods):
                        dct[attr] = func_logger(func)
            return type.__new__(cls, name, bases, dct)

    class TrackableObject(base):
        __metaclass__ = TrackableMeta

    return TrackableObject()

This basically uses a metaclass to override every method of an object to add some revision logging if the object changes. This is not super thoroughly tested and I haven't tried any other object types besides list and dict, but it seems to work okay for those.

Answer 2

Instead of monkey patching, you can create a proxy class:

• Make a proxy class that inherit from dict/list/set whatever
• Intercept attribute setting, and if the value is a dict/list/set, wrap it into the proxy class
• In proxy class __getattribute__, make sure the method is called on the wrapped type, but take care of tracking before doing so.

Pro:

• no class alteration

Con:

• you are limited to a number of types you know and expect