Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does multiprocessing.Queue have no task_done method

Why does Queue.Queue have a task_done method while multiprocessing.Queue has no such method?

like image 497
Baz Avatar asked Jun 09 '15 06:06

Baz


People also ask

What is queue Task_done ()?

Queue. task_done () Indicate that a formerly enqueued task is complete. Used by queue consumer threads. For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.

What is a multiprocessing queue?

A queue is a data structure on which items can be added by a call to put() and from which items can be retrieved by a call to get(). The multiprocessing. Queue provides a first-in, first-out FIFO queue, which means that the items are retrieved from the queue in the order they were added.

Is multiprocessing queue process safe?

Queues are thread and process safe.

Is multiprocessing faster than multithreading?

Multiprocessing outshines threading in cases where the program is CPU intensive and doesn't have to do any IO or user interaction. For example, any program that just crunches numbers will see a massive speedup from multiprocessing; in fact, threading will probably slow it down.


2 Answers

I think you need JoinableQueue.

like image 91
i.krivosheev Avatar answered Sep 19 '22 02:09

i.krivosheev


My guess would be: multithreading module was implemented very early, multiprocessing module came in 2.6 version.

The queue design was slightly corrected for multiprocessing and offers better flexibility than the multithreading, because you can choose between Queue, SimpleQueue and JoinableQueue depending on your use cases (speed vs reliability).

Now modifing multithreading like this would have caused backwards incompatibility, since join and task_done methods would have to be removed. Imagine the code needed to be refactored, new tests had to be written, API broken - for me clearly no benefits.

like image 31
matino Avatar answered Sep 21 '22 02:09

matino