Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python multithreaded print statements delayed until all threads complete execution

I have a piece of code below that creates a few threads to perform a task, which works perfectly well on its own. However I'm struggling to understand why the print statements I call in my function do not execute until all threads complete and the print 'finished' statement is called. I would expect them to be called as the thread executes. Is there any simple way to accomplish this, and why does this work this way in the first place?

def func(param):     time.sleep(.25)     print param*2  if __name__ == '__main__':     print 'starting execution'     launchTime = time.clock()     params = range(10)     pool=multiprocessing.Pool(processes=100) #use N processes to download the data     _=pool.map(func,params)     print 'finished' 
like image 411
Hawkwing Avatar asked Aug 14 '13 14:08

Hawkwing


People also ask

Does Python wait for threads to finish?

As in most programming languages, there are threads in Python too. Code executes sequentially, meaning that every function waits for the previous function to complete before it can execute.

Why Python is not good for multithreading?

Python doesn't support multi-threading because Python on the Cpython interpreter does not support true multi-core execution via multithreading. However, Python does have a threading library. The GIL does not prevent threading.

Is Python multithreading slow?

Test results. The test results show multi-threaded code is indeed significantly slower compared to multi-process code or even serialised execution.

How do you print a thread output in Python?

The print() function is a built-in function for printing a string on stdout and is not thread-safe. The print() function takes a string message, or an object that can be converted to a string.


2 Answers

For python 3 you can now use the flush param like that:

print('Your text', flush=True)

like image 120
Or Duan Avatar answered Sep 22 '22 19:09

Or Duan


This happens due to stdout buffering. You still can flush the buffers:

import sys  print 'starting' sys.stdout.flush() 

You can find more info on this issue here and here.

like image 24
Yossi Avatar answered Sep 22 '22 19:09

Yossi