How to create queues of objects in Django?

Question

I am new to Django and I am trying to build a blog myself. I'm trying to create a feature I've seen implemented in Drupal using the nodequeue module.

What I want to do is to be able to create queues of objects, for example, queues of blog posts. Below, I describe how I imagine the queues to work:

• the size of each queue should be user-defined.
• the date an object is added to a queue should be recorded.
• I would like to be able to define the order of the items that belong to each queue (but I think this would be very difficult).
• if the queue is full, the addition of an extra item should discard the oldest item of the queue.

An example of how such a feature would be useful is the creation of a featured posts queue.

My current knowledge does not allow me to figure out what would be the right way to do it. I would appreciate any pointers.

Thanks in advance

Answer 1

Here's one approach:

import collections, datetime, itertools

class nodequeue(object):
  def __init__(self, N):
    self.data = collections.deque(N * [(None, None)])
  def add(self, anobj):
    self.data.popleft()
    self.data.push((anobj, datetime.datetime.now())
  def __iter__(self):
    it = iter(self.data)
    return it.dropwhile(lambda x: x[1] is None, self.data)

This ignores the "ordering" desiderata, but that wouldn't be too hard to add, e.g.:

class nodequeueprio(object):
  def __init__(self, N):
    self.data = collections.deque(N * [(None, None, None)])
  def add(self, anobj, prio):
    self.data.popleft()
    self.data.push((anobj, datetime.datetime.now(), prio)
  def __iter__(self):
    it = iter(self.data)
    return sorted(it.dropwhile(lambda x: x[1] is None, self.data),
                  key=operator.itemgetter(2))

I think that prefilling the queue with placeholder Nones simplifies the code because add can always drop the leftmost (oldest or None) item before adding the new thingy -- even though __iter__ must then remove the placeholders, that's not too bad.

Answer 2

Alex's approach is great. I won't pretend to compete with his level of expertise, but for sake of completeness, here is another approach using the fantastic Queue.Queue class (bonus: thread-safe, but that's kinda useless to you based on your description). This might be easier to understand for you, as you expressed some concern on that point:

myqueue.py

#!/usr/bin/python

# Renamed in Python 3.0
try: from Queue import Queue, Full, Empty
except: from queue import Queue, Full, Empty
from datetime import datetime

# Spec 1: Size of each queue should be user-defined.
#   - maxsize on __init__

# Spec 2: Date an object is added should be recorded.
#   - datetime.now() is first member of tuple, data is second

# Spec 3: I would like to be able to define the order of the items that
# belong to each queue.
#   - Order cannot be rearranged with this queue.

# Spec 4: If the queue is full, the addition of an extra item should discard
# the oldest item of the queue.
#   - implemented in put()

class MyQueue(Queue):
    "Wrapper around Queue that discards old items instead of blocking."
    def __init__(self, maxsize=10):
        assert type(maxsize) is int, "maxsize should be an integer"
        Queue.__init__(self, maxsize)

    def put(self, item):
        "Put an item into the queue, possibly discarding an old item."
        try:
            Queue.put(self, (datetime.now(), item), False)
        except Full:
            # If we're full, pop an item off and add on the end.
            Queue.get(self, False)
            Queue.put(self, (datetime.now(), item), False)

    def put_nowait(self, item):
        "Put an item into the queue, possibly discarding an old item."
        self.put(item)

    def get(self):
        "Get a tuple containing an item and the datetime it was entered."
        try:
            return Queue.get(self, False)
        except Empty:
            return None

    def get_nowait(self):
        "Get a tuple containing an item and the datetime it was entered."
        return self.get()


def main():
    "Simple test method, showing at least spec #4 working."
    queue = MyQueue(5)
    for i in range(1, 7):
        queue.put("Test item number %u" % i)

    while not queue.empty():
        time_and_data = queue.get()
        print "%s => %s" % time_and_data


if __name__ == "__main__":
    main()

expected output

2009-11-02 23:18:37.518586 => Test item number 2
2009-11-02 23:18:37.518600 => Test item number 3
2009-11-02 23:18:37.518612 => Test item number 4
2009-11-02 23:18:37.518625 => Test item number 5
2009-11-02 23:18:37.518655 => Test item number 6