Can obj.__dict__ be inspected if the class has a __dict__ variable?

Question

I am interested in whether there is a way to introspect a Python instance infallibly to see its __dict__ despite any obstacles that the programmer might have thrown in the way, because that would help me debug problems like unintended reference loops and dangling resources like open files.

A simpler example is: how can I see the keys of a dict subclass if the programmer has hidden keys() behind a class of its own? The way around that is to manually call the dict keys() method instead of letting inheritance call the subclass's version of the method:

# Simple example of getting to the real info
# about an instance

class KeyHidingDict(dict):
    def keys(self):
        return []  # there are no keys here!

khd = KeyHidingDict(a=1, b=2, c=3)
khd.keys()       # drat, returns []
dict.keys(khd)   # aha! returns ['a', 'b', 'c']

Now my actual question is, how can I see the __dict__ of an instance, no matter what the programmer might have done to hide it from me? If they set a __dict__ class variable then it seems to shadow the actual __dict__ of any objects inherited from that class:

# My actual question

class DunderDictHider(object):
    __dict__ = {'fake': 'dict'}

ddh = DunderDictHider()
ddh.a = 1
ddh.b = 2
print ddh.a        # prints out 1
print ddh.__dict__ # drat, prints {'fake': 'dict'}

This false value for __dict__ does not, as you can see, interfere with actual attribute setting and getting, but it does mislead dir() by hiding a and b and displaying fake as the object's instance variable instead.

Again, my goal is to write a tool that helps me introspect class instances to see “what is really going on” when I am wondering why a set of class instances is taking so much memory or holding so many files open — and even though the situation above is extremely contrived, finding a way around it would let the tool work all the time instead of saying “works great, unless the class you are looking at has… [description of the exceptional situation above].”

I had thought I would be able to infallibly grab the __dict__ with something like:

dict_descr = object.__dict__['__dict__']
print dict_descr(ddh, DunderDictHider)

But it turns out that object does not have a __dict__ descriptor. Instead, the subtype_dict() C function seems to get separately attached to each subclass of object that the programmer creates; there is no central way to name or fetch the descriptor so that it can be manually applied to objects whose class shadows it.

Any ideas, anyone? :)

Answer 1

I'm not sure I'm happy with how simple this is:

>>> class DunderDictHider(object):
...     __dict__ = {'fake': 'dict'}
... 
>>> ddh = DunderDictHider()
>>> ddh.a = 1
>>> ddh.b = 2
>>> 
>>> print ddh.a
1
>>> print ddh.__dict__
{'fake': 'dict'}

The problem is that the class is cheating? Fix that!

>>> class DictUnhider(object):
...     pass
... 
>>> ddh.__class__ = DictUnhider
>>> print ddh.a
1
>>> print ddh.__dict__
{'a': 1, 'b': 2}

And there it is. This completely fails though, if the class defines any slots.

>>> class DoesntHaveDict(object):
...     __slots__ = ['a', 'b']
... 
>>> dhd = DoesntHaveDict()
>>> dhd.a = 1
>>> dhd.b = 2
>>> 
>>> print dhd.a
1
>>> dhd.__class__ = DictUnhider
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __class__ assignment: 'DoesntHaveDict' object layout differs from 'DictUnhider'
>>>