What is the reliable method to find most time consuming part of the code?

Question

Along my source code I try to capture and measure the time release of a segment in Python. How can I measure that segment pass time in a convenient way with good precision?

Answer 1

Use a profiler.

Python's cProfile is included in the standard libary.

For an even more convenient way, use the package profilestats. Then you can use a decorator to just decorate the functions you want to profile:

from profilestats import profile

@profile
def my_function(args, etc):
    pass

This will cause a summary like this to be printed on STDOUT:

         6 function calls in 0.026 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.026    0.026 some_code.py:3(some_func)
        2    0.019    0.010    0.026    0.013 some_code.py:9(expensive_func)
        2    0.007    0.003    0.007    0.003 {range}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

The much more useful info however is in the cachegrind.out.profilestats file generated. You can open this file with a tools that can visualize cachegrind statistics, for example RunSnakeRun and get nice, easy (or easier) to read visualizations of your call stack like this:

Update: Both pull requests for profilestats and pyprof2calltree have been merged, so they now support Python 3.x as well.