Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: Queue.Empty Exception Handling

Tags:

After a short debate with someone about exception handling in Python - sparked by the handling of a queue object - I thought I'd throw it out there...

METHOD 1:

import Queue  q = Queue.Queue()  try:     task=q.get(False)     #Opt 1: Handle task here and call q.task_done() except Queue.Empty:     #Handle empty queue here     pass  #Opt2: Handle task here and call q.task_done() 

METHOD 2:

import Queue  q = Queue.Queue()  if q.empty():     #Handle empty queue here else:     task = q.get()     #Handle task here     q.task_done() 

One argument is that Method 1 is wrong because the queue being empty is not an error, and therefore should not be handled using Queue.Empty exception. Additionally, it could make debugging more difficult when coded this way if you consider that the task handling part could potentially large.

The other argument is that either way is acceptable in Python and that handling the task outside of the try/except could aid debugging if task handling is large, although agreed that this might look uglier than using Method 2.

Opinions?

UPDATE: A little more info after answer 1 came through.... The debate was started after method 1 was using in some multithreaded code. In which case, the code will acquire the lock (from a threading.Lock object) and release it either once the task it returned or Queue.Empty is thrown

UPDATE 2: It was unknown to both of us that the the queue object was thread safe. Looks like try/except is the way to go!

like image 642
user1014903 Avatar asked Jun 28 '12 15:06

user1014903


People also ask

How do you clear a queue in Python?

To clear all items from a queue in Python:Call the clear() method on the deque object, e.g. q. queue. clear() . The clear method will remove all elements from the queue.

How do I know if my queue is empty?

queue::empty() is used to check whether the associated queue container is empty or not. This function returns either true or false, if the queue is empty (size is 0) then the function returns true, else if the queue is having some value then it will return false.

Is queue in Python thread-safe?

Thread Programming Luckily, Queue() class has a thread-safe implementation with all the required locking mechanism. So producer and consumer from different threads can work with the same queue instance safely and easily.


1 Answers

Method 2 is wrong because you are doing an operation in two steps when it could be done in one. In method 2, you check if the queue is empty, and then later (very soon, but still later), try to get the item. What if you have two threads pulling items from the queue? The get() could still fail with an empty queue. What if an item is added to the queue after you checked that it was empty? These are the sort of tiny windows of opportunity where bugs creep in to concurrent code.

Do it in one step, it's by far the better choice.

import Queue  q = Queue.Queue()  try:     task = q.get(False) except Queue.Empty:     # Handle empty queue here     pass else:     # Handle task here and call q.task_done() 

Don't get hung up on "exceptions should be errors". Exceptions are simply another channel of communication, use them. Use the "else" clause here to narrow the scope of the exception clause.

like image 114
Ned Batchelder Avatar answered Oct 29 '22 15:10

Ned Batchelder