Importing a local variable in a function into timeit

Question

I need to time the execution of a function across variable amounts of data.

def foo(raw_data):
   preprocessed_data = preprocess_data(raw_data)
   time = timeit.Timer('module.expensive_func(preprocessed_data)', 'import module').timeit()

However, preprocessed_data is not a global variable. It cannot be imported with from __main__. It is local to this subroutine.

How can i import data into the timeit.Timer environment?

Answer 1

Pass it a callable to time, rather than a string. (Unfortunately, this introduces some extra function call overhead, so it's only viable when the thing to time swamps that overhead.)

time = timeit.timeit(lambda: module.expensive_func(data))

In Python 3.5 and up, you can also specify an explicit globals dictionary with a string statement to time:

time = timeit.timeit('module.expensive_func(data)',
                     globals={'module': module, 'data': data})

Answer 2

The accepted answer didn't work for me inside pdb debugger and a class method. The solution that worked is to add the variables to globals():

globals()['data'] = data
globals()['self'] = self
timeit.timeit(lambda: self.function(data))

Note that the timing overhead is a little larger in this case because of the extra function calls. [source]