Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a "single slot" queue?

Tags:

python

queue

I need to use a queue which holds only one element, any new element discarding the existing one. Is there a built-in solution?

The solution I coded works but I strive not to reinvent the wheel :)

import Queue

def myput(q, what):
    # empty the queue
    while not q.empty():
        q.get()
    q.put(what)

q = Queue.Queue()
print("queue size: {}".format(q.qsize()))
myput(q, "hello")
myput(q, "hello")
myput(q, "hello")
print("queue size: {}".format(q.qsize()))

EDIT: following some comments & answers -- I know that a variable is just for that :) In my program, though, queues will be used to communicate between processes.

like image 366
WoJ Avatar asked Jun 30 '14 10:06

WoJ


2 Answers

As you specify you are using queues to communicate between processes, you should use the multiprocesssing.Queue.

In order to ensure there is only one item in the queue at once, you can have the producers sharing a lock and, whilst locked, first get_nowait from the queue before put. This is similar to the loop you have in your code, but without the race condition of two producers both emptying the queue before putting their new item, and therefore ending up with two items in the queue.

like image 136
Will Avatar answered Nov 10 '22 08:11

Will


Although the OP is regarding inter-process-communication, I came across a situation where I needed a queue with a single element (such that old elements are discarded when a new element is appended) set up between two threads (producer/consumer).

The following code illustrates the solution I came up with using a collections.deque as was mentioned in the comments:

import collections
import _thread
import time

def main():

    def producer(q):
       i = 0
       while True:
           q.append(i)
           i+=1
           time.sleep(0.75)


    def consumer(q):
        while True:
            try:
                v = q.popleft()
                print(v)
            except IndexError:
                print("nothing to pop...queue is empty")
                sleep(1)

    deq = collections.deque(maxlen=1)
    print("starting")
    _thread.start_new_thread(producer, (deq,))
    _thread.start_new_thread(consumer, (deq,))


if __name__ == "__main__":
    main()

In the code above, since the producer is faster than the consumer (sleeps less), some of the elements will not be processed.

Notes (from the documentation):

Deques support thread-safe, memory efficient appends and pops from either side of the deque with approximately the same O(1) performance in either direction.

Once a bounded length deque is full, when new items are added, a corresponding number of items are discarded from the opposite end.

Warning: The code never stops :)

like image 42
Daniel Avatar answered Nov 10 '22 09:11

Daniel