Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does the ipython magic function `%timeit -n1 code_block` execute `code_block` multiple times?

I am trying to run a particular test multiple times in ipython using the %timeit magic function. For demonstration purposes, I will just use -n1 instead of -n3 here, and use a simple print(1) function.

The %%timeit and the %timeit help says the following:

Options: -n<N>: execute the given statement <N> times in a loop. If this
value is not given, a fitting value is chosen.

-r<R>: repeat the loop iteration <R> times and take the best result.
Default: 3 (the 3 here is a typo in ipython, for which I have submitted a
PR)

However, if I do the following:

%%timeit -n1
print(1)

or

%timeit -n1 print(1)

it actually prints 1 7 times in a row as follows

In[1]: %timeit -n1 print(1)
1
1
1
1
1
1
1
32.8 µs ± 38.7 µs per loop (mean ± std. dev. of 7 runs, 1 loop each)

I was expecting that because of the definition of %%timeit/%timeit it would run the cell or code just once.

I have read this post: https://stackoverflow.com/a/45375047/4752883 which gives some examples of how %%timeit runs and the actual source code of ipython magic function %%timeit here: https://github.com/ipython/ipython/blob/ec3d1a11bf26a0962cb2cf55ba263b12ac023183/IPython/core/magics/execution.py#L944

where they define 2 types of loops: 1) -n<N> and 2) -r<R>.

If I just use -n1, it seems that it also assumes that I have used -r7, i.e. -n1 defaults to -n1 -r7. This means that even though I want it to run for exactly 1 run, it will still run the code_block 7 times as per https://github.com/ipython/ipython/blob/ec3d1a11bf26a0962cb2cf55ba263b12ac023183/IPython/core/magics/execution.py#L1021 unless I also specify -n1 -r1.

Questions:

  1. Why are there 2 different ways to running the code_block using -n<N> and -r<R>?
  2. What is the difference between -n<N> and -r<R> and why is this necessary?
like image 244
alpha_989 Avatar asked Mar 07 '23 00:03

alpha_989


1 Answers

These parameters are also in the timeit module.

  • -n determines how many times you run the function (or block, or whatever) inside the timing window. So the stopwatch starts, the code is run n times, then the stopwatch ends. You should run it enough times that the results are meaningful (timeit defaults to powers of 10 until 0.2 seconds have elapsed).
  • -r determines how many of these repetitions (where repetition is "start timer, run n times, stop timer") you should do. There is always some error due to your CPU scheduling other processes, etc., so usually you want to run it a few times and the best value of these r times is taken. (timeit defaults to 3, and the comments in the source you link indicate that ipython does the same -- but the actual code may disagree).

In pseudo-python, you can see how n and r affect the timing process:

time_hist = []
for _ in range(r):
    t0 = time.now()              # Start stopwatch (.now() is not a real function)
    for _ in range(n):
        # <your code block>
    t1 = time.now()              # Stop stopwatch

    time_hist.append(t1 - t0)    # Append time delta

 return min(time_hist)           # Return the min of the r deltas   
like image 127
jedwards Avatar answered Apr 27 '23 00:04

jedwards