Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Notification of key expiration in redis python

I would like to be notified when a volatile key expires in my redis store. The redis website provides some description of how this might be achieved in http://redis.io/topics/notifications, but I am wondering if it can be done using the python redis api.

After setting:notify-keyspace-events Ex in my redis.conf file

and running this as a test:

import redis import config  client = redis.StrictRedis.from_url(config.REDIS_URI)  client.set_response_callback('EXPIRE',callback) client.set('a', 1) client.expire('a',5) 

callback() only gets called when client.expire('a',5) gets called, but not five seconds later as expected

like image 601
samfr Avatar asked May 30 '14 23:05

samfr


People also ask

What happens when Redis key expires?

After the timeout has expired, the key will automatically be deleted. A key with an associated timeout is often said to be volatile in Redis terminology. The timeout will only be cleared by commands that delete or overwrite the contents of the key, including DEL , SET , GETSET and all the *STORE commands.

Can you define an expiration time of a field in a hash Redis?

Hi, it is not possible, either use a different top-level key for that specific field, or store along with the filed another field with an expire time, fetch both, and let the application understand if it is still valid or not based on current time.

What is Keyspace in Redis?

Redis keyspace notifications allow you to subscribe to PubSub channels. Through the channel, clients receive published events when a Redis command or data alteration occurs. These notifications are useful when an application must respond to changes that occur to the value stored in a particular key or keys.


1 Answers

The surprise (no expiration events seen when time to live for a key reaches zero) is not bound to Python, but rather to the way, Redis is expiring keys.

Redis doc on Timing of expired events

Timing of expired events

Keys with a time to live associated are expired by Redis in two ways:

  • When the key is accessed by a command and is found to be expired.
  • Via a background system that looks for expired keys in background, incrementally, in order to be able to also collect keys that are never accessed.

The expired events are generated when a key is accessed and is found to be expired by one of the above systems, as a result there are no guarantees that the Redis server will be able to generate the expired event at the time the key time to live reaches the value of zero.

If no command targets the key constantly, and there are many keys with a TTL associated, there can be a significant delay between the time the key time to live drops to zero, and the time the expired event is generated.

Basically expired events are generated when the Redis server deletes the key and not when the time to live theoretically reaches the value of zero.

Small test on console

when Redis running ($ sudo service redis-server start)

I started one console and have subscribed:

$ redis-cli PSUBSCRIBE "__key*__:*" 

Then, in another console:

$ redis-cli > config set notify-keyspace-events AKE 

what shall subscribe to all kinds of events

Then I continued with experiments in this second console:

> set aaa aaa > del aaa > set aaa ex 5 > get aaa 

All the activities were seen in subscribed console. Only the key expiration was sometime few seconds delayed, sometime came just in time.

Note alse, there are subtle differences in messages, one message __keyevent@0__:expire another __keyevent@0__:expired.

Sample listener spy.py

import redis import time  r = redis.StrictRedis() pubsub = r.pubsub() pubsub.psubscribe("*") for msg in pubsub.listen():     print time.time(), msg 

This code registers to all existing channels in default redis and prints whatever gets published.

Run it:

$ python spy.py 

and in another console try to set a key with an expiration. You will see all the events.

For following redis-cli input.

$ redis-cli 127.0.0.1:6379> set a aha OK 127.0.0.1:6379> set b bebe ex 3 OK 127.0.0.1:6379> set b bebe ex 3 OK 

we get spy output:

1401548400.27 {'pattern': None, 'type': 'psubscribe', 'channel': '*', 'data': 1L} 1401548428.36 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyspace@0__:a', 'data': 'set'} 1401548428.36 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyevent@0__:set', 'data': 'a'} 1401548436.8 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyspace@0__:b', 'data': 'set'} 1401548436.8 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyevent@0__:set', 'data': 'b'} 1401548436.8 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyspace@0__:b', 'data': 'expire'} 1401548436.8 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyevent@0__:expire', 'data': 'b'} 1401548439.82 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyspace@0__:b', 'data': 'expired'} 1401548439.82 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyevent@0__:expired', 'data': 'b'} 1401548484.46 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyspace@0__:b', 'data': 'set'} 1401548484.46 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyevent@0__:set', 'data': 'b'} 1401548484.46 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyspace@0__:b', 'data': 'expire'} 1401548484.46 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyevent@0__:expire', 'data': 'b'} 1401548487.51 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyspace@0__:b', 'data': 'expired'} 1401548487.51 {'pattern': '*', 'type': 'pmessage', 'channel': '__keyevent@0__:expired', 'data': 'b'} 
like image 139
Jan Vlcinsky Avatar answered Sep 20 '22 03:09

Jan Vlcinsky