MacOSX Instruments to profile Python code

Question

MacOSX Xcode Instruments is really great for profiling native code. However, I have also a huge chunk of Python calls in my callstacks. Can I somehow make Instruments Python-aware?

One solution I could think of is when it sees some PyEval_EvalFrameEx frame that it looks in its local variables / parameters to separate different Python calls and show me some call info.

I'm not asking about just a Python profiler. I want to profile my native code. But in this native code profiling, I want to add some further intelligence to analyze and translate the Python stack frames.

Answer 1

According to this stackoverflow answer, Instruments is a GUI front-end to dtrace. There is Apple Documentation confirming this and some OS-X specific articles on dtrace at Big Nerd Ranch among other places.

There are patches that can be applied to the CPython source before compiling it to instrument it for dtrace. It appears that there is or used to be support for automatically building a new python with dtrace in homebrew, but googling now, I'm not finding references for a homebrew recipe with dtrace provider support for current python releases (2.7.10, 3.4/3.5). I haven't tried, so maybe the current recipe just works with a --with-dtrace switch when building.

There is a talk from PyTexas 2013: dtrace, Python and You which talks about getting a python install with dtrace support included (specifically demonstrating with a Mac), and using dtrace on the command line.

I would think that once you had a python with dtrace support installed, when running it, you should be able to see and use it in Instruments. If you're adding a python interpreter to an OS X application (either as a .framework or some other form of linking), if that python had the dtrace patches applied before compilation, I would also think that it would be available to work with in dtrace. I've tried neither, but given what I know about dtrace, I believe it should work. If I confirm this is true, I will post back.

Answer 2

There is no MacOSX instrument to profile Python code. Personally, I use cProfile. Its an internal profiler called cProfile. You can use it in either of the ways below:

import cProfile cProfile.run('print "Hello World!"') 

or

python -m cProfile your_own_script.py 

The result would be something like:

>>> cProfile.run('print "Hello World!"') Hello World!          2 function calls in 0.000 seconds     Ordered by: standard name     ncalls  tottime  percall  cumtime  percall filename:lineno(function)         1    0.000    0.000    0.000    0.000 <string>:1(<module>)         1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects} 

Edit: If you are looking for native level calls. Then you should move to strace(for Linux) and dtruss (for Mac OSX) or someother tool like that. Basically you can wrap your Python script in an strace call and you will be able to view all your "native" C/C++ calls. For other linux systems use:

strace -fetrace=open python your_script.py 

or if you are on Mac OSX:

dtruss -f -t open python your_script.py 

I hope this helps!