Similar questions may have been asked a couple of times before, but none of them seem to have my case/scenario or it does not work.
I am trying to multithread a for loop as showed in an example. This for loop will do a function as it loops through an array. I would like to multithread it.
Example:
array = ["a", "b", "c", "d", "e"]
def dosomething(var):
#dosomething this is just an example as my actual code is not relevant to this question
for arrayval in array:
dosomething(arrayval)
This should loop through the array and do the function dosomething
with the variables a
, then b
, c
, etc.
Any idea on how I can do that?
To parallelize the loop, we can use the multiprocessing package in Python as it supports creating a child process by the request of another ongoing process. The multiprocessing module could be used instead of the for loop to execute operations on every element of the iterable. It's multiprocessing.
On a system with more than one processor or CPU cores (as is common with modern processors), multiple processes or threads can be executed in parallel.
Yes you can do multithreading on a single processor system. In multi-processor system , multiple threads execute , simultaneously on different cores. Eg- If there are two threads and two cores , then each thread would run on individual core.
You can use threading.Thread
:
from threading import Thread
from time import sleep
from random import randint
def dosomething(var):
sleep(randint(1,5))
print(var)
array = ["a", "b", "c", "d", "e"]
threads = []
for arrayval in array:
threads.append(Thread(target=dosomething, args=(arrayval,)))
threads[-1].start()
for thread in threads:
thread.join()
This outputs in random order within 5 seconds:
e
b
c
a
d
If you want to limit the number of threads you can use multiprocessing.pool.ThreadPool
instead. The following example limits the number of worker threads to 2 so it can possibly take as long as 15 seconds to complete (if the workers all happen to take 5 seconds):
from multiprocessing.pool import ThreadPool
from time import sleep
from random import randint
def dosomething(var):
sleep(randint(1,5))
print(var)
array = ["a", "b", "c", "d", "e"]
with ThreadPool(processes=2) as pool:
pool.map(dosomething, array)
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With