Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to synchronise multiple writer on redis?

I have multiple writers overwriting the same key in redis. How do I guarantee that only the chosen one write last?

Can I perform write synchronisation in Redis withour synchronise the writers first?


Background: In my system a unique dispatcher send works to do to various workers. Each worker then write the result in Redis overwrite the same key. I need to be sure that only the last worker that receive work from the dispatcher writes in Redis. 

like image 571
Lorenzo Belli Avatar asked Oct 18 '22 20:10

Lorenzo Belli


1 Answers

Use an ordered set (ZSET): add your entry with a score equal to the unix timestamp, then delete all but the top rank.

A Redis Ordered set is a set, where each entry also has a score. The set is ordered according to the score, and the position of an element in the ordered set is called Rank.

In order:

  1. Remove all the entries with score equal or less then the one you are adding(zremrangebyscore). Since you are adding to a set, in case your value is duplicate your new entry would be ignored, you want instead to keep the entry with highest rank. 
  2. Add your value to the zset (zadd)
  3. delete by rank all the entries but the one with HIGHEST rank (zremrangebyrank)
  4. You should do it inside a transaction (pipeline)

Example in python:

# timestamp contains the time when the dispatcher sent a message to this worker
key = "key_zset:%s"%id
pipeline = self._redis_connection.db.pipeline(transaction=True)
pipeline.zremrangebyscore(key, 0, t)  # Avoid duplicate Scores and identical data
pipeline.zadd(key, t, "value")
pipeline.zremrangebyrank(key, 0, -2)
pipeline.execute(raise_on_error=True)
like image 189
Lorenzo Belli Avatar answered Oct 21 '22 00:10

Lorenzo Belli