Add timeout argument to python's Queue.join()

Question

I want to be able to join() the Queue class but timeouting after some time if the call hasn't returned yet. What is the best way to do it? Is it possible to do it by subclassing queue\using metaclass?

Answer 1

Subclassing Queue is probably the best way. Something like this should work (untested):

def join_with_timeout(self, timeout):
    self.all_tasks_done.acquire()
    try:
        endtime = time() + timeout
        while self.unfinished_tasks:
            remaining = endtime - time()
            if remaining <= 0.0:
                raise NotFinished
            self.all_tasks_done.wait(remaining)
    finally:
        self.all_tasks_done.release()

Answer 2

The join() method is all about waiting for all the tasks to be done. If you don't care whether the tasks have actually finished, you can periodically poll the unfinished task count:

stop = time() + timeout
while q.unfinished_tasks and time() < stop:
    sleep(1)

This loop will exist either when the tasks are done or when the timeout period has elapsed.

Raymond