Lua and serialized closures

Question

I'm trying to serialize and deserialize a Lua closure

my basic understanding is that the below factory should generate closures (and that Lua doesn't much distinguish between functions and closures -- i.e. there is no type 'closure')

> function ffactory(x) return function() return x end end
> f1 = ffactory(5)
> print(f1())
5                        <-- so far so good
> s = string.dump(f1)
> f2 = load(s)
> print(f2())
table: 00000000002F7BA0  <-- expected the integer 5
> print(f2()==_ENV)
true                     <-- definitely didn't expect this!

I expected the integer 5 to be serialized with f1. Or, if string.dump can't handle closures, i expected an error.

I get quite different (but more what I expected) results with a mild change. It looks like f2 is indeed a closure, but string.dump didn't attempt to serialize the value of x at the time it was serialized.

The docs don't help me much. (what do they mean by "...with new upvalues"?)

> function ffactory(x) return function() return x+1 end end
> f1 = ffactory(5)
> print(f1())
6                        <-- good
> s = string.dump(f1)
> f2 = load(s)
> print(f2())
stdin:1: attempt to perform arithmetic on upvalue 'x' (a table value)
stack traceback:
        stdin:1: in function 'f2'
        stdin:1: in main chunk
        [C]: in ?

Answer 1

You can do something like this to save/restore those upvalues (note it doesn't handle upvalues shared between different functions):

local function capture(func)
  local vars = {}
  local i = 1
  while true do
    local name, value = debug.getupvalue(func, i)
    if not name then break end
    vars[i] = value
    i = i + 1
  end
  return vars
end

local function restore(func, vars)
  for i, value in ipairs(vars) do
    debug.setupvalue(func, i, value)
  end
end  

function ffactory(x) return function() return x end end
local f1 = ffactory(5)
local f2 = (loadstring or load)(string.dump(f1))
restore(f2, capture(f1)) --<-- this restored upvalues from f1 for f2

print(f1(), f2())

This works under both Lua 5.1 and Lua 5.2.

Note an interesting result if you change ffactory slightly (added math.abs(0); anything that uses global table in any way will do):

function ffactory(x) return function() math.abs(0) return x end end

Now if you restore upvalues you get the same result, but if you don't restore upvalues you get a run-time error under Lua 5.2:

lua.exe: upvalues.lua:19: attempt to index upvalue '_ENV' (a nil value)
stack traceback:
upvalues.lua:19: in function 'f2'
upvalues.lua:24: in main chunk
[C]: in ?

Answer 2

the docs are pretty clear. string.dump doesn't handle closures using upvalues. this is because the upvalues could be anything (including userdata, and how would Lua know how to serialize that?)

upvalues are the external variables that are local to a function due to scoping/closures. since x in your example is an upvalue to the function returned by ffactory, it does not get serialized.

if you want to support this somehow, you would have to store the upvalues yourself and set them again after you've deserialized the function, like this:

function ffactory(x)
    return function() return x+1 end
end

local f1 = ffactory(5)
print(f1())

local s = string.dump(f1)
f2 = loadstring(s)
debug.setupvalue(f2, 1, 5)
print(f2())