Python Executor - Passing function with argument as argument

Question

My goal is to create a function that I can use to measure the execution and resource use of another function. Using a tutorial, I've create the below using Python's ThreadPoolExecutor:

from resource import *
from time import sleep
from concurrent.futures import ThreadPoolExecutor

class MemoryMonitor:
    def __init__(self):
        self.keep_measuring = True

    def measure_usage(self):
        max_usage = 0
        u_run_time = 0
        s_run_time = 0
        while self.keep_measuring:
            max_usage = max(max_usage, getrusage(RUSAGE_SELF).ru_maxrss)
            u_run_time = max(u_run_time, getrusage(RUSAGE_SELF).ru_utime) 
            s_run_time = max(s_run_time, getrusage(RUSAGE_SELF).ru_stime) 
        sleep(0.1) # run this loop every 0.1 seconds
        return [max_usage, u_run_time, s_run_time]

def execute(function):
    with ThreadPoolExecutor() as executor:
        monitor = MemoryMonitor()
        stats_thread = executor.submit(monitor.measure_usage)
        try:
            fn_thread = executor.submit(function)
            result = fn_thread.result()
            print("print result")
            print(result)
            print("print result type")
            print(type(result))
        finally:
            monitor.keep_measuring = False
            stats = stats_thread.result()
        print(stats)
        return result

def foo():
    i = 0
    while i < 3:
        print("foo")
        i+=1
    return 1

def bar(x):
    while x < 3:
        print("foobar")
        x+=1
    return 1

var = execute(foo)
print("Var = " + str(var))
var = execute(bar(0))
print("Var = " + str(var))

If I pass the function foo as an argument to the function execute, it prints the correct results and returns the value returned by foo.

If I pass the function bar in the same way, but with bar itself requiring an argument, the function runs (prints 3 times) and then I get the following error:

result = self.fn(*self.args, **self.kwargs)
TypeError: 'int' object is not callable

After some testing, the part where I'm stuck appears to be passing a function as an argument, if that function itself requires an argument. As I understand the ThreadPoolExecutor, the fn_thread object encapsulates the execution of the function submitted. The result object should simply hold the result of that execution - what am I missing that this cannot handle being passed a function with an argument?

Answer 1

You are submitting

bar(0) 

instead of

bar, 0

To clarify, look at the submit's signature: submit(fn, *args, **kwargs)

the result of

bar(0)

is an integer, and the executor cannot call an integer, since it is not 'callable', as the error message suggests.