Can Python be made to generate tracing similar to bash's set -x?

Question

Is there a similar mechanism in Python, to the effect set -x has on bash?

Here's some example output from bash in this mode:

+ for src in cpfs.c log.c popcnt.c ssse3_popcount.c blkcache.c context.c types.c device.c
++ my_mktemp blkcache.c.o
+++ mktemp -t blkcache.c.o.2160.XXX
++ p=/tmp/blkcache.c.o.2160.IKA
++ test 0 -eq 0
++ echo /tmp/blkcache.c.o.2160.IKA
+ obj=/tmp/blkcache.c.o.2160.IKA

I'm aware of the Python trace module, however its output seems to be extremely verbose, and not high level like that of bash.

Answer 1

Perhaps use sys.settrace:

Use traceit() to turn on tracing, use traceit(False) to turn off tracing.

import sys
import linecache

def _traceit(frame, event, arg):
    '''
    http://www.dalkescientific.com/writings/diary/archive/2005/04/20/tracing_python_code.html
    '''
    if event == "line":
        lineno = frame.f_lineno
        filename = frame.f_globals["__file__"]
        if (filename.endswith(".pyc") or
            filename.endswith(".pyo")):
            filename = filename[:-1]
        name = frame.f_globals["__name__"]
        line = linecache.getline(filename, lineno)
        print "%s  # %s:%s" % (line.rstrip(), name, lineno,)
    return _traceit

def _passit(frame, event, arg):
    return _passit

def traceit(on=True):
    if on: sys.settrace(_traceit)
    else: sys.settrace(_passit)

def mktemp(src):
    pass

def my_mktemp(src):
    mktemp(src)
    p=src

traceit()
for src in ('cpfs.c','log.c',):
    my_mktemp(src)
traceit(False)

yields

mktemp(src)  # __main__:33
pass  # __main__:30
p=src  # __main__:34
mktemp(src)  # __main__:33
pass  # __main__:30
p=src  # __main__:34
if on: sys.settrace(_traceit)  # __main__:26
else: sys.settrace(_passit)  # __main__:27

Answer 2

To trace specific calls, you can wrap each interesting function with your own logger. This does lead to arguments expanded to their values rather than just argument names in the output.

Functions have to be passed in as strings to prevent issues where modules redirect to other modules, like os.path / posixpath. I don't think you can extract the right module name to patch from just the function object.

Wrapping code:

import importlib

def wrapper(ffull, f):
    def logger(*args, **kwargs):
        print "TRACE: %s (%s, %s)" % (ffull, args, kwargs)
        return f(*args, **kwargs)
    return logger

def log_execution(ffull):
    parts = ffull.split('.')
    mname = '.'.join(parts[:-1])
    fname = parts[-1]
    m = importlib.import_module(mname)
    f = getattr(m, fname)
    setattr(m, fname, wrapper(ffull, f))

Usage:

for f in ['os.path.join', 'os.listdir', 'sys.exit']:
    log_execution(f)

p = os.path.join('/usr', 'bin')
os.listdir(p)
sys.exit(0)

....

% ./a.py  
TRACE: os.path.join (('/usr', 'bin'), {})
TRACE: os.listdir (('/usr/bin',), {})
TRACE: sys.exit ((0,), {})