Why does Python's queue.Queue.get() permit returning early from timeouts?

Question

UPDATE: This question is based on a faulty mental model of how Queue.get() was actually behaving, which was caused by some slightly ambiguous documentation but mainly by a buggy, hand-rolled implementation of timedelta.total_seconds(). I discovered this bug when trying to prove that the original answers where incorrect. Now that timedelta.total_seconds() is provided by Python (since 2.7), I will move to use that.

Sorry for the confusion.

---

This isn't a "Why does my code not run?" question, but a "What is the motivation behind this design decision?"

Since 2.3, Python's queue module contains a Queue class with a get method, that takes a timeout parameter. Here's the section from the manual:

Queue.get([block[, timeout]])
Remove and return an item from the queue. If optional args block is true and timeout is None (the default), block if necessary until an item is available. If timeout is a positive number, it blocks at most timeout seconds and raises the Empty exception if no item was available within that time. [...]

(Emphasis mine)

Note that it may raise an Empty exception even if it hasn't reached the timeout. In fact, I am seeing that behaviour on Ubuntu (but not Windows). It is bailing just a little early and it has had minor consequences on my code - I can code around it though.

Most blocking timeouts take a minimum timeout, which makes sense on a non-real-time OS, including Windows and Linux. There is no guarantee that the OS will context switch to your process or thread by any given deadline.

However, this one takes a maximum timeout. Can anyone explain how this design decision might make sense?

Answer 1

I think you are misinterpreting the documentation. It isn't saying it might raise the Empty exception after less than timeout seconds, it is saying it will block for at most timeout seconds. It might block for less than that, if it can satisfy the get.

I realize you are saying you see it raising Empty early, but honestly, that sounds like either a bug, or that you are relying on more accuracy than the system can provide. (It does seem as though, to obey the exact wording of the specification, an implementation should round timeout down to the resolution of its timer rather than up as you seem to desire.)