Let's say I have two processes: a reader and a writer. How does the writer detect when the reader has finished writing values?
The multiprocessing
module has a queue with a close method that seems custom-built for this purpose. But how do you detect when the queue has been closed?
This doesn't seem to work, as the writer never exits:
import multiprocessing as mp
def getter(q):
while True:
try:
print(q.get())
except Exception:
break
def putter(q):
q.put(1)
q.put(2)
q.close()
q = mp.Queue()
writer = mp.Process(target=putter, args=(q, ))
reader = mp.Process(target=getter, args=(q, ))
reader.start()
writer.start()
writer.join()
reader.join()
Should the reader-writer pair use a sentinel value to signal end of writing? Then what's the point of having the close
method?
EDIT: While this question asks about the Queue module (now queue), I am asking specifically about mp.Queue
and what the correct use of the .close
method is.
The empty() method checks if a multiprocessing queue is empty if the method returns True if the queue is empty. Otherwise, it returns False .
Queues are thread and process safe.
To clear all items from a queue in Python:queue. clear() . The clear method will remove all elements from the queue.
There is no direct way of clearing a multiprocessing. Queue . I believe the closest you have is close() , but that simply states that no more data will be pushed to that queue, and will close it when all data has been flushed to the pipe.
But how do you detect when the queue has been closed?
You don't. That is not the purpose of close
. Calling close
doesn't even guarantee that no more items will be added to the queue; the docs say
Indicate that no more data will be put on this queue by the current process.
close
is intended to shut down the current process's feeder thread for that queue (or at least start shutting it down), not to communicate an end-of-queue to other processes.
If you want to signal that no more values will be written to the queue, use standard techniques like enqueueing a sentinel object, like you would with an ordinary queue.Queue
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With