Use Python's `timeit` from a program but functioning the same way as the command line?

Question

For instance, documentation says:

Note however that timeit will automatically determine the number of repetitions only when the command-line interface is used.

Is there a way to call it from within a Python script and have the number of repetitions be determined automatically, with only the shortest number returned?

Answer 1

When you call timeit from the command line like this:

python -mtimeit -s'import test' 'test.foo()'

The timeit module is called as a script. In particular, the main function is called:

if __name__ == "__main__":
    sys.exit(main())

If you look at the source code, you'll see that the main function can take an args argument:

def main(args=None):    
    if args is None:
        args = sys.argv[1:]

So indeed it is possible to run timeit from within a program with exactly the same behavior as you see when run from the CLI. Just supply your own args instead of allowing it to be set to sys.argv[1:]:

import timeit
import shlex

def foo():
    total = 0
    for i in range(10000):
        total += i**3
    return total

timeit.main(args=shlex.split("""-s'from __main__ import foo' 'foo()'"""))

will print something like

100 loops, best of 3: 7.9 msec per loop

---

Unfortunately, main prints to the console, instead of returning the time per loop. So if you want to programmatically use the result, perhaps the easiest way would be to start by copying the main function and then modifying it -- changing the printing code to instead return usec.

---

Example by OP: If you place this in utils_timeit.py:

import timeit
def timeit_auto(stmt="pass", setup="pass", repeat=3):
    """
    http://stackoverflow.com/q/19062202/190597 (endolith)
    Imitate default behavior when timeit is run as a script.

    Runs enough loops so that total execution time is greater than 0.2 sec,
    and then repeats that 3 times and keeps the lowest value.

    Returns the number of loops and the time for each loop in microseconds
    """
    t = timeit.Timer(stmt, setup)

    # determine number so that 0.2 <= total time < 2.0
    for i in range(1, 10):
        number = 10**i
        x = t.timeit(number) # seconds
        if x >= 0.2:
            break
    r = t.repeat(repeat, number)
    best = min(r)
    usec = best * 1e6 / number
    return number, usec

you can use it in scripts like this:

import timeit
import utils_timeit as UT

def foo():
    total = 0
    for i in range(10000):
        total += i**3
    return total

num, timing = UT.timeit_auto(setup='from __main__ import foo', stmt='foo()')
print(num, timing)

Answer 2

As of Python 3.6, timeit.Timer objects have an autorange function that exposes how number is determined for command line execution.