Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Blocking dict in Python?

Is there a data structure in Python that resembles a blocking dictionary? This data structure must fulfill these requirements:

  • it must be randomly accessible and allow any element to be modified/deleted (not just the first or last)
  • it must have a blocking get() and put()
  • it must be thread-safe

I would have used a queue but, although blocking and thread-safe, it's not randomly accessible. A dict is not blocking either (as far as my Python knowledge goes). As an example, think of one producer thread adding key-value pairs to such a data-structure (updating values for existing keys if already present - this is where a queue won't cut it), and a worker blocking on get() and consuming these key-value pairs as they become available. Many many thanks!

edit: Let's assume the producer polls a CI server and gets project-status pairs. It generates the differences in project statuses and puts them in the aforementioned data structure. The worker picks up these project-status updates and displays them one by one as an animation on the screen.

class Producer:
  def generateProjectStatusChanges():
    ...
  def updateSuperAwesomeDataStructure(changes):
    for (proj, stat) in changes:
      #queue won't do cause the update could take place in the middle of the queue
      #hence the dict behavior
      superAwesomeDS.putOrUpdate(proj, stat) 
  def watchForUpdates():
     changes = generateProjectStatusChanges()
     updateSuperAwesomeDataStructure(changes)
     time.sleep(self.interval)

class Worker:
  def blockingNotifyAnimation():
    ...
  def watchForUpdates():
    while true:
      proj, stat = superAwesomeDS.getFirstPair() #or any pair really
      blockingNotifyAnimation(proj, stat)
like image 213
George Flourentzos Avatar asked Nov 01 '22 15:11

George Flourentzos


1 Answers

Something along the following lines should do the trick (untested):

class UpdatableBlockingQueue(object):

  def __init__(self):
    self.queue = {}
    self.cv = threading.Condition()

  def put(self, key, value):
    with self.cv:
      self.queue[key] = value
      self.cv.notify()

  def pop(self):
    with self.cv:
      while not self.queue:
        self.cv.wait()
      return self.queue.popitem()

It uses a dictionary for the queue and a condition variable for serialising access and signalling between threads.

like image 145
NPE Avatar answered Nov 09 '22 22:11

NPE