redis block until key exists

Question

I'm new to Redis and was wondering if there is a way to be able to await geting a value by it's key until the key exists. Minimal code:

async def handler():
    data = await self._fetch(key)

async def _fetch(key):
    return self.redis_connection.get(key)

As you know, if such key doesnt exist, it return's None. But since in my project, seting key value pair to redis takes place in another application, I want the redis_connection get method to block untill key exists. Is such expectation even valid?

Answer 1

It is not possible to do what you are trying to do without implementing some sort of polling redis GET on your client. On that case your client would have to do something like:

async def _fetch(key):
    val = self.redis_connection.get(key)
    while val is None:
       # Sleep and retry here
       asyncio.sleep(1)  
       val = self.redis_connection.get(key)
    return val

However I would ask you to completelly reconsider the pattern you are using for this problem. It seems to me that what you need its to do something like Pub/Sub https://redis.io/topics/pubsub.

So the app that performs the SET becomes a publisher, and the app that does the GET and waits until the key is available becomes the subscriber.

I did a bit of research on this and it looks like you can do it with asyncio_redis:

• Subscriber https://github.com/jonathanslenders/asyncio-redis/blob/b20d4050ca96338a129b30370cfaa22cc7ce3886/examples/pubsub/receiver.py.

• Sender(Publisher): https://github.com/jonathanslenders/asyncio-redis/blob/b20d4050ca96338a129b30370cfaa22cc7ce3886/examples/pubsub/sender.py

Hope this helps.

Answer 2

The closest you can get to this behavior is by enabling keyspace notifications and subscribing to the relevant channels (possibly by pattern).

Note, however, that notifications rely on PubSub that is not guaranteed to deliver messages (at-most-once semantics).

Answer 3

Except the keyspace notification method mentioned by @Itamar Haber, another solution is the blocking operations on LIST.

1. handler method calls BRPOP on an empty LIST: BRPOP notify-list timeout, and blocks until notify-list is NOT empty.
2. The other application pushes the value to the LIST when it finishes setting the key-value pair as usual: SET key value; LPUSH notify-list value.
3. handler awake from the blocking operation with the value you want, and the notify-list is destroyed by Redis automatically.

The advantage of this solution is that you don't need to modify your handler method too much (with the keyspace notification solution, you need to register a callback function). While the disadvantage is that you have to rely on the notification of another application (with keyspace notification solution, Redis does the notification automatically).