Menu
I have developed a python C-extension that receives data from python and compute some cpu intensive calculations. It's possible to profile the C-extension?
The problem here is that writing a sample test in C to be profiled would be challenging because the code rely on particular inputs and data structures (generated by python control code).
Do you have any suggestions?
564
Any code that you write using any compiled language like C, C++, or Java can be integrated or imported into another Python script. This code is considered as an "extension." A Python extension module is nothing more than a normal C library.
Extending Python with C or C++ It is quite easy to add new built-in modules to Python, if you know how to program in C. Such extension modules can do two things that can't be done directly in Python: they can implement new built-in object types, and they can call C library functions and system calls.
Profiling Small Fragments In Python's standard library, we have the timeit module that allows us to do some simple profiling. The output of timeit is to find the best performance among multiple runs (default to be 5). Each run is to run the provided statements a few times (which is dynamically determined).
The syntax is cProfile. run(statement, filename=None, sort=-1) . You can pass python code or a function name that you want to profile as a string to the statement argument. If you want to save the output in a file, it can be passed to the filename argument.
After the comment by pygabriel I decided to upload a package to pypi that implements a profiler for python extensions using the cpu-profiler from google-perftools: http://pypi.python.org/pypi/yep
165
I've found my way using google-perftools. The trick was to wrap the functions StartProfiler and StopProfiler in python (throught cython in my case).
To profile the C extension is sufficient to wrap the python code inside the StartProfiler and StopProfiler calls.
from google_perftools_wrapped import StartProfiler, StopProfiler import c_extension # extension to profile c_extension.so StartProfiler("output.prof") ... calling the interesting functions from the C extension module ... StopProfiler() Then to analyze for example you can export in callgrind format and see the result in kcachegrind:
pprof --callgrind c_extension.so output.prof > output.callgrind kcachegrind output.callgrind If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With