Using cProfile results with KCacheGrind

Question

With cProfile you can also profile existing programs, without making any separate profiling script. Just run program with profiler

python -m cProfile -o profile_data.pyprof script_to_profile.py

and open profile data in kcachegrind with pyprof2calltree, whose -k switch automatically opens data in kcachegrind

pyprof2calltree -i profile_data.pyprof -k

For example profiling whole paster server and webapp would be done like this

python -m cProfile -o pyprof.out `which paster` serve development.ini

pyprof2calltree can be installed with easy_install.

You could use profilestats.profile decorator ($ pip install profilestats) -- a simple wrapper for pyprof2calltree module (rebranding of lsprofcalltree.py):

from profilestats import profile

@profile
def func():
    # do something here

Script can be run as usual. profilestats creates two files: cachegrind.out.profilestats and profilestats.prof in KCachegrind-compatible and cProfile formats correspondingly.

It can be done using an external module called lscallproftree

This article explains how: CherryPy - CacheGrind

With my resulting code looking like so:

...
if profile:
    import cProfile
    import lsprofcalltree

    profileFileName = 'Profiles/pythonray_' + time.strftime('%Y%m%d_%H%M%S') + '.profile'

    profile = cProfile.Profile()
    profile.run('pilImage = camera.render(scene, samplePattern)')

    kProfile = lsprofcalltree.KCacheGrind(profile)

    kFile = open (profileFileName, 'w+')
    kProfile.output(kFile)
    kFile.close()

    profile.print_stats()    
else:            
    pilImage = camera.render(scene, samplePattern)
...

If anyone knows a way to do this that doesn't require an external (ie. not shipped with Python) module, I'd still be very interested to hear about it.

3 differents ways to profile your code and visualizing results in KCachegrind/Qcachegrind:

I - CPROFILE

1 - Profile myfunc() from ipython

import cProfile
filename = 'filename.prof'
cProfile.run('myfunc()', filename)

2 - Convert your file to a usable kcachegrind file in your shell

sudo pip install pyprof2calltree
pyprof2calltree -i filename.prof -o callgrind.filename.prof

3 - Open callgrind.filename.prof in kcachegrind

II - EMBEDDED CPROFILE

1 - Profile few lines in your code.

import cProfile
filename = 'filename.prof'
pr = cProfile.Profile()
pr.enable()
# ... lines to profile ...
pr.disable()
pr.dump_stats(filename)

2 - Convert your file to a usable kcachegrind file in your shell

sudo pip install pyprof2calltree
pyprof2calltree -i filename.prof -o callgrind.filename.prof

3 - Open callgrind.filename.prof in kcachegrind

III - YAPPI

1 - Profile myfunc() from ipython or from your code

import yappi
filename = 'callgrind.filename.prof'
yappi.set_clock_type('cpu')
yappi.start(builtins=True)
myfunc()
stats = yappi.get_func_stats()
stats.save(filename, type='callgrind')

2 - Open callgrind.filename.prof in kcachegrind