How do I use line_profiler (from Robert Kern)?

Question

I have tried using the line_profiler module for getting a line-by-line profile over a Python file. This is what I've done so far:

1) Installed line_profiler from pypi by using the .exe file (I am on WinXP and Win7). Just clicked through the installation wizard.

2) Written a small piece of code (similar to what has been asked in another answered question here).

from line_profiler import LineProfiler def do_stuff(numbers):     print numbers  numbers = 2 profile = LineProfiler(do_stuff(numbers)) profile.print_stats() 

3) Run the code from IDLE/PyScripter. I got only the time.

Timer unit: 4.17188e-10 s 

How do I get full line-by-line profile over the code I execute? I have never used any advanced Python features like decorators, so it is hard for me to understand how shall I use the guidelines provided by several posts like here and here.

Answer 1

This answer is a copy of my answer here for how to get line_profiler statistics from within a Python script (without using kernprof from the command line or having to add @profile decorators to functions and class methods). All answers (that I've seen) to similar line_profiler questions only describe using kernprof.

---

The line_profiler test cases (found on GitHub) have an example of how to generate profile data from within a Python script. You have to wrap the function that you want to profile and then call the wrapper passing any desired function arguments.

from line_profiler import LineProfiler import random  def do_stuff(numbers):     s = sum(numbers)     l = [numbers[i]/43 for i in range(len(numbers))]     m = ['hello'+str(numbers[i]) for i in range(len(numbers))]  numbers = [random.randint(1,100) for i in range(1000)] lp = LineProfiler() lp_wrapper = lp(do_stuff) lp_wrapper(numbers) lp.print_stats() 

Output:

Timer unit: 1e-06 s  Total time: 0.000649 s File: <ipython-input-2-2e060b054fea> Function: do_stuff at line 4  Line #      Hits         Time  Per Hit   % Time  Line Contents ==============================================================      4                                           def do_stuff(numbers):      5         1           10     10.0      1.5      s = sum(numbers)      6         1          186    186.0     28.7      l = [numbers[i]/43 for i in range(len(numbers))]      7         1          453    453.0     69.8      m = ['hello'+str(numbers[i]) for i in range(len(numbers))] 

Adding Additional Functions to Profile

Also, you can add additional functions to be profiled as well. For example, if you had a second called function and you only wrap the calling function, you'll only see the profile results from the calling function.

from line_profiler import LineProfiler import random  def do_other_stuff(numbers):     s = sum(numbers)  def do_stuff(numbers):     do_other_stuff(numbers)     l = [numbers[i]/43 for i in range(len(numbers))]     m = ['hello'+str(numbers[i]) for i in range(len(numbers))]  numbers = [random.randint(1,100) for i in range(1000)] lp = LineProfiler() lp_wrapper = lp(do_stuff) lp_wrapper(numbers) lp.print_stats() 

The above would only produce the following profile output for the calling function:

Timer unit: 1e-06 s  Total time: 0.000773 s File: <ipython-input-3-ec0394d0a501> Function: do_stuff at line 7  Line #      Hits         Time  Per Hit   % Time  Line Contents ==============================================================      7                                           def do_stuff(numbers):      8         1           11     11.0      1.4      do_other_stuff(numbers)      9         1          236    236.0     30.5      l = [numbers[i]/43 for i in range(len(numbers))]     10         1          526    526.0     68.0      m = ['hello'+str(numbers[i]) for i in range(len(numbers))] 

In this case, you can add the additional called function to profile like this:

from line_profiler import LineProfiler import random  def do_other_stuff(numbers):     s = sum(numbers)  def do_stuff(numbers):     do_other_stuff(numbers)     l = [numbers[i]/43 for i in range(len(numbers))]     m = ['hello'+str(numbers[i]) for i in range(len(numbers))]  numbers = [random.randint(1,100) for i in range(1000)] lp = LineProfiler() lp.add_function(do_other_stuff)   # add additional function to profile lp_wrapper = lp(do_stuff) lp_wrapper(numbers) lp.print_stats() 

Output:

Timer unit: 1e-06 s  Total time: 9e-06 s File: <ipython-input-4-dae73707787c> Function: do_other_stuff at line 4  Line #      Hits         Time  Per Hit   % Time  Line Contents ==============================================================      4                                           def do_other_stuff(numbers):      5         1            9      9.0    100.0      s = sum(numbers)  Total time: 0.000694 s File: <ipython-input-4-dae73707787c> Function: do_stuff at line 7  Line #      Hits         Time  Per Hit   % Time  Line Contents ==============================================================      7                                           def do_stuff(numbers):      8         1           12     12.0      1.7      do_other_stuff(numbers)      9         1          208    208.0     30.0      l = [numbers[i]/43 for i in range(len(numbers))]     10         1          474    474.0     68.3      m = ['hello'+str(numbers[i]) for i in range(len(numbers))] 

NOTE: Adding functions to profile in this way does not require changes to the profiled code (i.e., no need to add @profile decorators).