Python: practical introspection

Question

I often find myself using third party libraries - packages and modules - that lack sufficient documentation. Studying the sourcecode therefore becomes essential, but can also be a somewhat tedious task. I (as I guess everybody) use dir() and help() functions to get started and recently I have begun using the inspect module. I would like to know what are the "methods" that you use to dive into badly documented code and how to increase efficiency in doing so. Help much appreciated.

Answer 1

I find IPython indispensable for this sort of task. The ? (show docstring) and ?? (show source) magic commands, coupled with IPython's excellent completion system and live object introspection really make a difference for me.

An example session:

In [1]: import sphinx.writers <TAB>
# see available modules and packages - narrow down

In [1]: import shpinx.writers.manpage as manpage
In [2]: manpage.<TAB>
# list and complete on the module's contents 

In [3]: manpage.Writer?
# nicely formatted docstring follows

In [4]: manpage.Writer??
# nicely formatted source code follows

In [5]: %edit manpage
# open module in editor
# it really helps if you use something like ctags at this point

In [6]: %edit manpage.Writer
# open module in editor - jump to class Writer

Unfortunately, not all code can be inspected this way. Think of projects that do things in modules without wrapping them in an if __name__ == '__main__' or projects that rely heavily on magic (sh comes to mind).

Answer 2

I'd like to construct callgraphs with http://pycallgraph.slowchop.com/ or doxygen.

In fact, AST module and some else allow for relatively painless statical analysis. What I'd like more is to somehow perform dynamical analysis (because, the value of what is called "func1" can change and calls may become completely different).