Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python Queue Count of Unfinished Tasks

I am implementing Python Queue with Thread. I need a way to get the total count of items currently in the queue (which I can get with queue.qsize()) and the count of unfinished tasks. Basically I need a count of all items that are being processed/need to be processed. The Python Queue documentation mentions the following:

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks.

But it offers no insight as to how to access that count. Thanks in advance!

like image 331
Mocking Avatar asked Apr 15 '16 23:04

Mocking


People also ask

Is Asyncio Threadsafe a queue?

Although asyncio queues are not thread-safe, they are designed to be used specifically in async/await code.

How do I find the length of a queue in Python?

To get the length of a queue in Python: Use the len() function to get the length of a deque object. Use the qsize() method to get the length of a queue object.

Are Python queues thread-safe?

Yes, Queue is thread-safe.


1 Answers

I'm surprised it is not mentioned in the documentation but Queue objects have a unfinished_tasks attribute:

>>> x = queue.Queue()
>>> x.unfinished_tasks
0
>>> x.put("test")
>>> x.unfinished_tasks
1

Since it is not mentioned in the documentation it isn't guaranteed to be the same attribute updating to newer versions (although I somewhat doubt it) and particularly I wouldn't trust this to be thread-safe (since any internal use would already be inside a lock acquire so external access likely doesn't)

like image 59
Tadhg McDonald-Jensen Avatar answered Oct 03 '22 07:10

Tadhg McDonald-Jensen