Inconsistency between %time and %timeit in IPython

Question

I am confronted to a weird situation that I can't explain. Here is my test timing the generation of a large list of tuples:

In [1]: def get_list_of_tuples():    ...:     return [(i,) for i in range(10**6)]    ...:  In [2]: %time res = get_list_of_tuples() CPU times: user 0.93 s, sys: 0.08 s, total: 1.01 s Wall time: 0.98 s  In [3]: %timeit res = get_list_of_tuples() 1 loops, best of 3: 92.1 ms per loop 

As you can see, the generation of this large list of tuples takes just below a second. timeit reports the execution time to be around 0.1 second. Why is there such a big difference in the two reports?

(Tested on IPython 0.11, Python 2.6.5.)

Answer 1

The main difference is because "by default, timeit() temporarily turns off garbage collection during the timing".

Turning the garbage collection returns results similar to the one shown in the question, i.e. the time of execution with garbage collection is magnitude bigger than the one without:

In [1]: import timeit  # Garbage collection on. In [2]: N = 10; timeit.timeit('[(i,) for i in range(10**6)]', 'gc.enable()', number=N) / N Out[2]: 0.74884700775146484 # 749 ms per loop.  # Garbage collection off. In [3]: N = 10; timeit.timeit('[(i,) for i in range(10**6)]', number=N) / N Out[3]: 0.15906109809875488 # 159 ms per loop.