Profiling a system with extensively reused decorators

Question

Our code base has a few decorators that are used extensively.

When I create a runtime profile, a large part of the call graph looks like an hour glass; many functions call one function (the decorator), which then calls many functions. This is a less-useful profile than I'd like.

Is there any way to rectify this situation? Removing the decorator is not an option; it provides necessary functionality.

We've considered manually stripping the decorator from the cProfile data after the fact, but it doesn't seem possible, because the data is summarized into caller->callee relationships, which destroys the caller->decorator->callee relationship.

Answer 1

Using something like the new library (or types in Python 2.6+), you could theoretically dynamically create a code object and then a function object based on that code object that had a built-in name that varied along with the function you were wrapping.

That would allow you to manipulate things as deep as <func>.__code__.co_name (which is normally read-only).

---

import functools
import types

def metadec(func):

    @functools.wraps(func)
    def wrapper(*args, **kwargs):   
        # do stuff
        return func(*args, **kwargs)

    c = wrapper.func_code
    fname = "%s__%s" % (func.__name__, wrapper.__name__)

    code = types.CodeType(
                c.co_argcount, 
                c.co_nlocals,
                c.co_stacksize,
                c.co_flags,  
                c.co_code,        
                c.co_consts,         
                c.co_names,
                c.co_varnames,
                c.co_filename,
                fname, # change the name
                c.co_firstlineno,
                c.co_lnotab,
                c.co_freevars,
                c.co_cellvars,
            )

    return types.FunctionType(
            code, # Use our updated code object
            wrapper.func_globals,
            fname, # Use the updated name
            wrapper.func_defaults,
            wrapper.func_closure,
        )

(functools.wraps is still used here in order to allow for pass-through of things like docstrings, module names, etc.)

---

In [1]: from metadec import metadec

In [2]: @metadec
   ...: def foobar(x):
   ...:     print(x)
   ...:     
   ...:     

In [3]: foobar.__name__
Out[3]: 'foobar__wrapper'

In [4]: foobar(1)
1