Executor.map on concurrent futures is not processed

Question

I cannot understand why a simple Executor won't work

I have two modules: main.py

import other

a = [1, 2, 3]
b = ['a', 'b'] 
l = ['a', 'b', 'd']

other.run_executor(a, b, l)

And other.py

from concurrent import futures
from multiprocessing import cpu_count


def do_stuff(a, b, c):
    print(a, b, c)


def run_executor(a, b, l):
    iterat = ((a, b, c) for c in l)
    with futures.ThreadPoolExecutor(max_workers=cpu_count() - 1) as e:
        e.map(do_stuff, iterat)

Running this code raises no error, but also does not print anything. It shoud print a, b, l[0]; a, b, l[1]; a, b, l[2] What am I doing wrong?

Answer 1

This is because you are not using ThreadPoolExecutor.map properly. The arguments should not be passed as one iterator but as 3 iterators:

from concurrent import futures
from itertools import repeat
from multiprocessing import cpu_count


def do_stuff(a, b, c):
    print(a, b, c)


def run_executor(a, b, l):
    with futures.ThreadPoolExecutor(max_workers=cpu_count() - 1) as e:
        e.map(do_stuff, repeat(a), repeat(b), c)

Also, you should use the if __name__ == "__main__" protection in your main script to avoid weird behaviors. This permit to protect your module main from executing if you try to import/pickle a function defined in it.

import other

if __name__ == "__main__":
    a = [1, 2, 3]
    b = ['a', 'b'] 
    l = ['a', 'b', 'd']

    other.run_executor(a, b, l)