Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Simple multithread for loop in Python

I searched everywhere and I don't find any simple example of iterating a loop with multithreading.

For example, how can I multithread this loop?

for item in range(0, 1000):
    print(item)

Is there any way to cut it in like 4 threads, so each thread has 250 iterations?

like image 474
Anthony Avatar asked Aug 09 '16 16:08

Anthony


People also ask

How do you use multithreading in a for loop?

n=100; // number of times to loop y=zeros(n,1); // holds results for j (1,n,1); x = rndu(10,1); // some data to analyze y[j,1] = somefunction(x); endfor; Execution could be speeded up if the statement in the loop were to run as independent threads.

How do you write multithread in Python?

The most common way to create a multithreaded python application is to declare a class which extends the Thread class and overrides it's run() method. The Thread class, in summary, signifies a code sequence that runs in a separate thread of control.

Is Python good for multi threading?

Python doesn't support multi-threading because Python on the Cpython interpreter does not support true multi-core execution via multithreading. However, Python does have a threading library. The GIL does not prevent threading.

How do you do a multiprocess loop in Python?

Parallel For-Loop with map() First, we can create the multiprocessing pool configured by default to use all available CPU cores. Next, we can call the map() function as before and iterate the result values, although this time the map() function is a method on the multiprocessing pool.


1 Answers

Easiest way is with multiprocessing.dummy (which uses threads instead of processes) and a Pool

import multiprocessing.dummy as mp 

def do_print(s):
    print s

if __name__=="__main__":
    p=mp.Pool(4)
    p.map(do_print,range(0,10)) # range(0,1000) if you want to replicate your example
    p.close()
    p.join()

Maybe you want to try real multiprocessing, too if you want to better utilize multiple CPUs but there are several caveats and guidelines to follow then.

Possibly other methods of Pool would better suit your needs - depending on what you are actually trying to do.

like image 56
janbrohl Avatar answered Oct 01 '22 06:10

janbrohl