Menu
I'm writing a custom sub-class of Queue.Queue and run into a situation where I need to acquire a queue-level lock when my custom put() is being called. I'd like to reuse the existing lock Queue has (Queue.mutex), but can't because its not an RLock.
In the source (python 2.6), it says:
# Override these methods [_put, _get, etc] to implement other queue organizations
# (e.g. stack or priority queue).
# These will only be called with appropriate locks held
But the online documentation doesn't mention overriding them. The other Queue implementations in that module override these. So, I'm kinda inclined to believe that the _put method is package-private and isn't really intended for use outside the Queue module.
Does anyone know how kosher it would be to use Queue._put and friends in my own subclass?
360
Queue in Python can be implemented using deque class from the collections module.
Thread Programming Luckily, Queue() class has a thread-safe implementation with all the required locking mechanism. So producer and consumer from different threads can work with the same queue instance safely and easily.
You can see that the claim of O(1) performance for deque is well founded, while a list is over twice as slow around 1,000 items, an order of magnitude around 10,000.
Queue's don't inherently have the idea of being complete or done. They can be used indefinitely. To close it up when you are done, you will indeed need to put None or some other magic value at the end and write the logic to check for it, as you described. The ideal way would probably be subclassing the Queue object.
As far as I'm concerned, it would be completely kosher. In Python, the source is the documentation, or at least should be considered as a supplement to the published API documentation. This is largely a consequence of the enforced use of whitespace and coding conventions that emphasize clear, readable code: when you have questions that the documentation doesn't answer, you're supposed to be able to go to the source code and look up the answers.
In particular, details like this one are irrelevant to most clients of the Queue module (who are simply using the classes), so they wouldn't make it into the published docs. But if you want to subclass Queue, the developers expect you to be digging deeper, so the note is there in the source code for you to find.
I will also mention that Python doesn't really have a concept of "package-private." Semantically, there's no difference between a subclass which is in the same module and one which is in a different module, even one you write yourself. In fact, Python doesn't even have a concept of "private". Instead, it relies on a principle of responsible usage: as a developer you're expected to be smart enough not to use internal methods when you don't need to. Starting a member name with an underscore is simply a clue that it's an internal method, and that you probably shouldn't be accessing it if you are simply using the class - but if you're subclassing it, anything is fair game.
140
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
