Modifying a C++ array in main() from Lua without extra allocation

Question

I am sketching a small C++ program that will pass arrays to Lua and have them modified there, where I intend to have a lua script read in the program so I can modify it without needing to recompile the program

My first obstacle is to ensure Lua is able to modify arrays already allocated instead of having them allocated again in the Lua space. The data will be float and the size will be really large, but I am starting small for the moment.

To simplify this interface I tried LuaBridge 2.6, but it doesn't provide the expected result. Below is a fully "working" program.

#include <iostream>
#include <cstdint>
#include <cstring>
#include <vector>
#include <lua5.3/lua.hpp>
#include <LuaBridge/LuaBridge.h>

int main(void)
    {
    const uint32_t      LENGTH = 512 * 256;
    std::vector <float> input(LENGTH),
                        output(LENGTH);

    memset(output.data(), 0, LENGTH * sizeof(float));   // Zero the output
    for(uint32_t i = 0; i < LENGTH; i++)                // Populate input
        input[i] = (float)i + 0.5f;

    lua_State *luastate = luaL_newstate();
    luabridge::push(luastate, input.data());    // Supposedly passing a pointer to the first element of input, according to LuaBridge manual chap 3-3.1
    luabridge::push(luastate, output.data());   // Same for output

    luaL_dostring(luastate, "output[10] = input[256]");     // Expecting to assign this value in the C++ arrays, not in the Lua space
    lua_getglobal(luastate, "output[10]");                  // Find this assigned value in the Lua stack
    lua_Number val = lua_tonumber(luastate, 1);             // Retrieving this value from Lua to C++

    std::cout << input[256] << ", " << output[10] << ", " << val << std::endl;  // The values of val and in output[10] don't match

    lua_close(luastate);

    return 0;
    }

Notice that nothing matches. What is going to output[10] in Lua is not the value of input[256] in the C++ space, but input[0]. The C++ output array is not updated from within Lua, cout shows that it remains as we initialized (0). To confirm that, we pushed this value of output[10] to the stack, which is not input[256] in C++, and retrieved from C++. Can you guys correct me or point me to where I should be going to achieve this?

======= UPDATE 08/11/2020 =======

To clarify what the program is doing (or supposed to do), after reading Robert's and Joseph's considerations, I post below an updated version of both the C++ part and the lua script called by it. Notice I abandoned LuaBridge since I didn't succeed in the first attempt:

C++:

#include <iostream>
#include <cstdint>
#include <cstring>
#include <vector>
#include <luajit-2.0/lua.hpp>  // LuaJIT 2.0.4 from Ubuntu 16.04

int main(void)
    {
    const uint32_t      LENGTH = 256 * 512;
    std::vector <float> input(LENGTH),
                        output(LENGTH);

    memset(output.data(), 0, LENGTH * sizeof(float));
    for(uint32_t i = 0; i < LENGTH; i++)
        input[i] = (float)i + 0.5f;

    lua_State *luastate = luaL_newstate();
    luaL_openlibs(luastate);

    // Here I have to pass &input[0], &output[0] and LENGTH
    // to Lua, which in turn will pass to whatever functions
    // are being called from a .so lib opened in Lua-side

    luaL_dofile(luastate, "my_script.lua");    
    lua_close(luastate);

    return 0;
    }

The Lua script looks like this:

local ffi = require("ffi")
local mylib = ffi.load("/path_to_lib/mylib.so")

-- Here I import and call any fuctions needed from mylib.so
-- without needing to recompile anything, just change this script
-- At this point the script has to know &input[0], &output[0] and LENGTH

ffi.cdef[[int func1(const float *in, float *out, const uint32_t LEN);]]
ffi.cdef[[int func2(const float *in, float *out, const uint32_t LEN);]]
ffi.cdef[[int funcX(const float *in, float *out, const uint32_t LEN);]]

if(mylib.func1(input, output, LENGTH) == 0) then
    print("Func1 ran successfuly.")
else
    print("Func1 failed.")
end

Answer 1

I am sketching a small C++ program that will pass arrays to Lua

The data will be float and the size will be really large,

My suggestion:

• Keep the buffer on the C side (as a global variable for example)
• Expose a C-function to LUA GetTableValue(Index)
• Expose a C-function to Lua SetTableValue(Index, Value)

It should be something like this:

static int LUA_GetTableValue (lua_State *LuaState)
{
  float Value;

  /* lua_gettop returns the number of arguments */
  if ((lua_gettop(LuaState) == 1) && (lua_isinteger(LuaState, -1)))
  {
    /* Get event string to execute (first parameter) */
    Offset = lua_tointeger(LuaState, -1);

    /* Get table value */
    Value  = LUA_FloatTable[Offset];

    /* Push result to the stack */
    lua_pushnumber(Value);
  }
  else
  {
    lua_pushnil(LuaState);  
  }

  /* return 1 value */
  return 1;
}

And you also need to register the function:

lua_register(LuaState, "GetTableValue", LUA_GetTableValue);

I let you write the SetTableValue but it should be very close. Doing so, the buffer is on C side and can be accessed from Lua with dedicated functions.

Answer 2

I recommend you create a userdata that exposes the arrays via __index and __newindex, something like this (written as a C and C++ polyglot like Lua itself):

#include <stdio.h>
#include <string.h>

#ifdef __cplusplus
extern "C" {
#endif
#include <lua5.3/lua.h>
#include <lua5.3/lauxlib.h>
#ifdef __cplusplus
}
#endif

struct MyNumbers {
    lua_Number *arr;
    lua_Integer len;
};

int MyNumbers_index(lua_State *L) {
    struct MyNumbers *t = (struct MyNumbers *)luaL_checkudata(L, 1, "MyNumbers");
    lua_Integer k = luaL_checkinteger(L, 2);
    if(k >= 0 && k < t->len) {
        lua_pushnumber(L, t->arr[k]);
    } else {
        lua_pushnil(L);
    }
    return 1;
}

int MyNumbers_newindex(lua_State *L) {
    struct MyNumbers *t = (struct MyNumbers *)luaL_checkudata(L, 1, "MyNumbers");
    lua_Integer k = luaL_checkinteger(L, 2);
    if(k >= 0 && k < t->len) {
        t->arr[k] = luaL_checknumber(L, 3);
        return 0;
    } else {
        return luaL_argerror(L, 2,
                             lua_pushfstring(L, "index %d out of range", k));
    }
}

struct MyNumbers *MyNumbers_new(lua_State *L, lua_Number *arr, lua_Integer len) {
    struct MyNumbers *var = (struct MyNumbers *)lua_newuserdata(L, sizeof *var);
    var->arr = arr;
    var->len = len;
    luaL_setmetatable(L, "MyNumbers");
    return var;
}

int main(void) {
    const lua_Integer LENGTH = 512 * 256;
    lua_Number input[LENGTH], output[LENGTH];

    memset(output, 0, sizeof output);
    for(lua_Integer i = 0; i < LENGTH; ++i)
        input[i] = i + 0.5f;

    lua_State *L = luaL_newstate();

    luaL_newmetatable(L, "MyNumbers");
    lua_pushcfunction(L, MyNumbers_index);
    lua_setfield(L, -2, "__index");
    lua_pushcfunction(L, MyNumbers_newindex);
    lua_setfield(L, -2, "__newindex");
    /* exercise for the reader: implement __len and __pairs too, and maybe shift the indices so they're 1-based to Lua */
    lua_pop(L, 1);

    MyNumbers_new(L, input, LENGTH);
    lua_setglobal(L, "input");
    MyNumbers_new(L, output, LENGTH);
    lua_setglobal(L, "output");

    luaL_dostring(L, "output[10] = input[256]");
    lua_getglobal(L, "output");
    lua_geti(L, -1, 10);
    lua_Number val = lua_tonumber(L, -1);

    printf("%f, %f, %f\n", input[256], output[10], val);

    lua_close(L);
}

With this approach, there is no copy of any data in Lua, and your own MyNumbers_ functions control how all access to them is done.

---

If you want to be able to use the arrays through LuaJIT's FFI instead of directly manipulating them in Lua, then you can pass their addresses in a light userdata instead, like this:

#include <string.h>

#ifdef __cplusplus
extern "C" {
#endif
#include <luajit-2.0/lua.h>
#include <luajit-2.0/lualib.h>
#include <luajit-2.0/lauxlib.h>
#ifdef __cplusplus
}
#endif

int main(void) {
    const lua_Integer LENGTH = 256 * 512;
    lua_Number input[LENGTH], output[LENGTH];

    memset(output, 0, sizeof output);
    for(lua_Integer i = 0; i < LENGTH; ++i)
        input[i] = i + 0.5f;

    lua_State *L = luaL_newstate();
    luaL_openlibs(L);

    lua_pushlightuserdata(L, input);
    lua_setglobal(L, "input");
    lua_pushlightuserdata(L, output);
    lua_setglobal(L, "output");
    lua_pushinteger(L, LENGTH);
    lua_setglobal(L, "LENGTH");

    luaL_dofile(L, "my_script.lua");
    lua_close(L);
}