Use Multiple DBs With One Redis Lua Script?

Question

Is it possible to have one Redis Lua script hit more than one database? I currently have information of one type in DB 0 and information of another type in DB 1. My normal workflow is doing updates on DB 1 based on an API call along with meta information from DB 0. I'd love to do everything in one Lua script, but can't figure out how to hit multiple dbs. I'm doing this in Python using redis-py:

lua_script(keys=some_keys,
           args=some_args,
           client=some_client)

Since the client implies a specific db, I'm stuck. Ideas?

Answer 1

It is usually a wrong idea to put related data in different Redis databases. There is almost no benefit compared to defining namespaces by key naming conventions (no extra granularity regarding security, persistence, expiration management, etc ...). And a major drawback is the clients have to manually handle the selection of the correct database, which is error prone for clients targeting multiple databases at the same time.

Now, if you still want to use multiple databases, there is a way to make it work with redis-py and Lua scripting.

redis-py does not define a wrapper for the SELECT command (normally used to switch the current database), because of the underlying thread-safe connection pool implementation. But nothing prevents you to call SELECT from a Lua script.

Consider the following example:

$ redis-cli
SELECT 0
SET mykey db0
SELECT 1
SET mykey db1

The following script displays the value of mykey in the 2 databases from the same client connection.

import redis

pool = redis.ConnectionPool(host='localhost', port=6379, db=0)
r = redis.Redis(connection_pool=pool)

lua1 = """
   redis.call("select", ARGV[1])
   return redis.call("get",KEYS[1])
"""
script1 = r.register_script(lua1)

lua2 = """
   redis.call("select", ARGV[1])
   local ret = redis.call("get",KEYS[1])
   redis.call("select", ARGV[2])
   return ret
"""
script2 = r.register_script(lua2)

print r.get("mykey")
print script2( keys=["mykey"], args = [1,0] )
print r.get("mykey"), "ok"
print
print r.get("mykey")
print script1( keys=["mykey"], args = [1] )
print r.get("mykey"), "misleading !!!"

Script lua1 is naive: it just selects a given database before returning the value. Its usage is misleading, because after its execution, the current database associated to the connection has changed. Don't do this.

Script lua2 is much better. It takes the target database and the current database as parameters. It makes sure that the current database is reactivated before the end of the script, so that next command applied on the connection still run in the correct database. Unfortunately, there is no command to guess the current database in the Lua script, so the client has to provide it systematically. Please note the Lua script must reset the current database at the end whatever happens (even in case of previous error), so it makes complex scripts cumbersome and awkward.