How to multi-thread with "for" loop?

Question

Similar questions may have been asked a couple of times before, but none of them seem to have my case/scenario or it does not work.

I am trying to multithread a for loop as showed in an example. This for loop will do a function as it loops through an array. I would like to multithread it.

Example:

array = ["a", "b", "c", "d", "e"]
def dosomething(var):
    #dosomething this is just an example as my actual code is not relevant to this question

for arrayval in array:
    dosomething(arrayval)

This should loop through the array and do the function dosomething with the variables a, then b, c, etc.

Any idea on how I can do that?

Answer 1

You can use threading.Thread:

from threading import Thread
from time import sleep
from random import randint

def dosomething(var):
    sleep(randint(1,5))
    print(var)

array = ["a", "b", "c", "d", "e"]
threads = []
for arrayval in array:
    threads.append(Thread(target=dosomething, args=(arrayval,)))
    threads[-1].start()
for thread in threads:
    thread.join()

This outputs in random order within 5 seconds:

e
b
c
a
d

If you want to limit the number of threads you can use multiprocessing.pool.ThreadPool instead. The following example limits the number of worker threads to 2 so it can possibly take as long as 15 seconds to complete (if the workers all happen to take 5 seconds):

from multiprocessing.pool import ThreadPool
from time import sleep
from random import randint

def dosomething(var):
    sleep(randint(1,5))
    print(var)

array = ["a", "b", "c", "d", "e"]
with ThreadPool(processes=2) as pool:
    pool.map(dosomething, array)