python time measure for every function [duplicate]

Question

I just completed writing my first program in python, i have written all my functions in a single module, i just executed it from command line by giving the input files as argument and it worked. But when i gave a big dataset, my program continuously running for some time. Now my next step is to find which function is taking more time in my module. I could get the time taken by whole program, but i need for each function separately.

I tried to understand timeit and profile modules in python, but as per my understanding, they were giving the time taken by a particular function. Is there a way to know the time taken by each function in my module as a statistics( all at once )?

Thanks in advance.

Answer 1

At the terminal, run

python -m profile -s time file.py

or

python -m cProfile -s time file.py

The second can be faster, and is never worse.

This will give something like:

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       39    0.132    0.003    0.139    0.004 :0(load_dynamic)
      239    0.097    0.000    0.097    0.000 :0(loads)
    541/1    0.083    0.000    3.556    3.556 :0(exec)
       30    0.082    0.003    0.082    0.003 :0(statusBar)
                        ... etc ...

The left hand side will contain your functions.

Answer 2

first i suggest use profilers module or timeit for this aim .timeit provides a simple way to time small bits of Python code !

To profile a function that takes a single argument, you can do:

import cProfile
import re
cProfile.run('re.compile("foo|bar")')

Also you can use a Decorator like this that allows you to measure the execution times of dedicated methods :

import time                                                

def measure_time(f):

  def timed(*args, **kw):
    ts = time.time()
    result = f(*args, **kw)
    te = time.time()

    print '%r (%r, %r) %2.2f sec' % \
          (f.__name__, args, kw, te-ts)
    return result

return timed

You can use it like this :

  @measure_time
  def foo():
        #content of function 

note that f.__name__ return the name of function ! (in this case 'foo')