Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

is it possible to call lua functions defined in other lua scripts in redis?

Tags:

node.js

redis

lua

I have tried to declare a function without the local keyword and then call that function from anther script but it gives me an error when I run the command.

test = function ()    
    return 'test'
end



# from some other script
test()

Edit:

I can't believe I still have no answer to this. I'll include more details of my setup.

I am using node with the redis-scripto package to load the scripts into redis. Here is an example.

var Scripto = require('redis-scripto');
var scriptManager = new Scripto(redis);

scriptManager.loadFromDir('./lua_scripts');

var keys    = [key1, key2];
var values  = [val];

scriptManager.run('run_function', keys, values, function(err, result) {
console.log(err, result)
})

And the lua scripts.

-- ./lua_scripts/dict_2_bulk.lua

-- turns a dictionary table into a bulk reply table
dict2bulk = function (dict)
    local result = {}
    for k, v in pairs(dict) do
        table.insert(result, k)
        table.insert(result, v)
    end
    return result
end


-- run_function.lua 

return dict2bulk({ test=1 })

Throws the following error.

[Error: ERR Error running script (call to f_d06f7fd783cc537d535ec59228a18f70fccde663): @enable_strict_lua:14: user_script:1: Script attempted to access unexisting global variable 'dict2bulk' ] undefined
like image 709
Tim Fairbrother Avatar asked Dec 15 '22 01:12

Tim Fairbrother


2 Answers

I'm going to be contrary to the accepted answer, because the accepted answer is wrong.

While you can't explicitly define named functions, you can call any script that you can call with EVALSHA. More specifically, all of the Lua scripts that you have explicitly defined via SCRIPT LOAD or implicitly via EVAL are available in the global Lua namespace at f_<sha1 hash> (until/unless you call SCRIPT FLUSH), which you can call any time.

The problem that you run into is that the functions are defined as taking no arguments, and the KEYS and ARGV tables are actually globals. So if you want to be able to communicate between Lua scripts, you either need to mangle your KEYS and ARGV tables, or you need to use the standard Redis keyspace for communication between your functions.

127.0.0.1:6379> script load "return {KEYS[1], ARGV[1]}"
"d006f1a90249474274c76f5be725b8f5804a346b"
127.0.0.1:6379> eval "return f_d006f1a90249474274c76f5be725b8f5804a346b()" 1 "hello" "world"
1) "hello"
2) "world"
127.0.0.1:6379> eval "KEYS[1] = 'blah!'; return f_d006f1a90249474274c76f5be725b8f5804a346b()" 1 "hello" "world"
1) "blah!"
2) "world"
127.0.0.1:6379>

All of this said, this is in complete violation of spec, and is entirely possible to stop working in strange ways if you attempt to run this in a Redis cluster scenario.

like image 193
Josiah Avatar answered May 15 '23 14:05

Josiah


Important Notice: See Josiah's answer below. My answer turns out to be wrong or at the least incomplete. Which makes me very happy ofcourse, it makes Redis all the more flexible.

My incorrect/incomplete answer:

I'm quite sure this is not possible. You are not allowed to use global variables (read the docs ), and the script itself gets a local and temporary scope by the Redis Lua engine.

Lua functions automatically set a 'writing' flag behind the scenes if they do any write action. This starts a transaction. If you cascade Lua calls, the bookkeeping in Redis would become very cumbersome, especially when the cascade is executed on a Redis slave. That's why EVAL and EVALSHA are intentionally not made available as valid Redis calls inside a Lua script. Same goes for calling an already 'loaded' Lua function which you are trying to do. What would happen if the slave is rebooted between the load of the first script and the exec of the second script?

What we do to overcome this limitation:

Don't use EVAL, only use SCRIPT LOAD and EVALSHA. Store the SHA1 inside a redis hash set.

We automated this in our versioning system, so a committed Lua script automatically gets it's SHA1 checksum stored in the Redis master, in a hash set, with a logical name. The clients can't use EVAL (on a slave; we disabled EVAL+LOAD in config). But the client can ask for the SHA1 for the next step. Almost all our Lua functions return a SHA1 for the next call.

Hope this helps, TW

like image 27
Tw Bert Avatar answered May 15 '23 12:05

Tw Bert