Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Tool to inspect Python objects without changing them

Tags:

python

While trying to track down a resource leak in a Python program this evening, it occurred to me that modern ORMs make the job quite difficult. An object which is, in fact, sitting alone in memory with no children will suddenly appear to have a dozen associated objects as you start checking its attributes because, of course, each attribute dereference invokes a descriptor that pulls in additional information on-the-fly.

I even noticed that doing a simple print of one particular object wound up doing a database query and pulling more linked objects into memory — ruining the careful reference counts that I had been computing — because its __repr__() built the displayed name out of a few associated objects.

There are, it happens, a few techniques that allow objects to be inspected without affecting them — operations like type(obj) and id(obj) and obj.__dict__. (But not printing the __dict__, since that invokes __repr__() on every single value in the dictionary!) Has anyone ever combined these few “safe” inspection methods to support, at a prompt like the Python debugger, convenient inspection and exploration of a Python object graph so that I can see where these files are being held open, running me out of file descriptors?

I need, essentially, an anti-Heisenberg tool, that prevents my acts of inspection from having any side effects!

The “inspect” module:

One answer suggests that I try the inspect() module, but it looks like it dereferences every attribute on the object you supply:

import inspect
class Thing(object):
    @property
    def one(self):
        print 'one() got called!'
        return 1
t = Thing()
inspect.getmembers(t)

This outputs:

one() got called!
[('__class__', <class '__main__.Thing'>),
 ('__delattr__', <method-wrapper '__delattr__'…),
 …
 ('one', 1)]
like image 672
Brandon Rhodes Avatar asked May 12 '11 03:05

Brandon Rhodes


1 Answers

Python 3.2 now provides inspect.getattr_static() for precisely this kind of use case: http://docs.python.org/py3k/library/inspect#fetching-attributes-statically

The source code link from the top of the docs page should make it fairly easy to backport that functionality to earlier versions (although keep in mind that as 3.x stdlib code, it isn't built to handle old-style classes).

I'm not aware of any existing tools that combine that kind of technique with inspection of obj.__dict__ to navigate a whole object graph without invoking descriptors, though.

like image 170
ncoghlan Avatar answered Oct 20 '22 11:10

ncoghlan