how to handle session expire basing redis?

Question

I want to implement a session store based on Redis. I would like to put session data into Redis. But I don't know how to handle session-expire. I can loop through all the redis keys (sessionid) and evaluate the last access time and max idle time, thus I need to load all the keys into the client, and there may be 1000m session keys and may lead to very poor I/O performances.
I want to let Redis manage the expire, but there are no listener or callback when the key expire, so it is impossible to trigger HttpSessionListener. Any advice?

Answer 1

So you need your application to be notified when a session expires in Redis.

While Redis does not support this feature, there are a number of tricks you can use to implement it.

Update: From version 2.8.0, Redis does support this http://redis.io/topics/notifications

First, people are thinking about it: this is still under discussion, but it might be added to a future version of Redis. See the following issues:

• https://github.com/antirez/redis/issues/83
• https://github.com/antirez/redis/issues/594

Now, here are some solutions you can use with the current Redis versions.

Solution 1: patching Redis

Actually, adding a simple notification when Redis performs key expiration is not that hard. It can be implemented by adding 10 lines to the db.c file of Redis source code. Here is an example:

https://gist.github.com/3258233

This short patch posts a key to the #expired list if the key has expired and starts with a '@' character (arbitrary choice). It can easily be adapted to your needs.

It is then trivial to use the EXPIRE or SETEX commands to set an expiration time for your session objects, and write a small daemon which loops on BRPOP to dequeue from the "#expired" list, and propagate the notification in your application.

An important point is to understand how the expiration mechanism works in Redis. There are actually two different paths for expiration, both active at the same time:

• Lazy (passive) mechanism. The expiration may occur each time a key is accessed.

• Active mechanism. An internal job regularly (randomly) samples a number of keys with expiration set, trying to find the ones to expire.

Note that the above patch works fine with both paths.

The consequence is Redis expiration time is not accurate. If all the keys have expiration, but only one is about to be expired, and it is not accessed, the active expiration job may take several minutes to find the key and expired it. If you need some accuracy in the notification, this is not the way to go.

Solution 2: simulating expiration with zsets

The idea here is to not rely on the Redis key expiration mechanism, but simulate it by using an additional index plus a polling daemon. It can work with an unmodified Redis 2.6 version.

Each time a session is added to Redis, you can run:

MULTI
SET <session id> <session content>
ZADD to_be_expired <current timestamp + session timeout> <session id>
EXEC

The to_be_expired sorted set is just an efficient way to access the first keys that should be expired. A daemon can poll on to_be_expired using the following Lua server-side script:

local res = redis.call('ZRANGEBYSCORE',KEYS[1], 0, ARGV[1], 'LIMIT', 0, 10 )
if #res > 0 then
   redis.call( 'ZREMRANGEBYRANK', KEYS[1], 0, #res-1 )
   return res
else
   return false
end

The command to launch the script would be:

EVAL <script> 1 to_be_expired <current timestamp>

The daemon will get at most 10 items. For each of them, it has to use the DEL command to remove the sessions, and notify the application. If one item was actually processed (i.e. the return of the Lua script is not empty), the daemon should loop immediately, otherwise a 1 second wait state can be introduced.

Thanks to the Lua script, it is possible to launch several polling daemons in parallel (the script guarantees that a given session will only be processed once, since the keys are removed from to_be_expired by the Lua script itself).

Solution 3: use an external distributed timer

Another solution is to rely on an external distributed timer. The beanstalk lightweight queuing system is a good possibility for this

Each time a session is added in the system, the application posts the session ID to a beanstalk queue with a delay corresponding to the session time out. A daemon is listening to the queue. When it can dequeue an item, it means a session has expired. It just has to clean the session in Redis, and notify the application.