How does Luabind work?

Question

I'm interested on how Luabind wrapper make it possible to pass a function without the lua_State *L and not using the Lua stack.

How does Luabind:

1. count the function parameters?
2. link the function parameters to Lua stack?
3. link those classes

I'm not trying to create another binding like Luabind to other libraries. I'm just wondering how did they do that. Just a curious man.

Answer 1

Good question. I had some vague idea of how luabind does what it does, but I didn't know enough to answer fully and accurately. Armed with an IDE and debugger I started dissecting the following very simple piece:

struct C
{
    int i;
    int f(int x, const char* s)
};

    lua_State* L = luaL_newstate();

open(L);
module(L)
[
    class_<C>("C")
        .def_readwrite("index", &C::i)
        .def("f", &C::f)
];

First thing to notice is that L is passed to luabind a lot, the call to open creates a few globals in the Lua state: the __luabind_classes of type userdata and two functions class and property. Luabind doesn't seem to use global variables - everything it needs is saved in the lua environment.

Now we get to module(L)[...]. The original code is the best explanation, first here's module:

inline module_ module(lua_State* L, char const* name = 0)
{
    return module_(L, name);
}

Simple enough, here's module_:

class LUABIND_API module_
{
public:
    module_(lua_State* L_, char const* name);
    void operator[](scope s);

private:
    lua_State* m_state;
    char const* m_name;
};

So what our little program does is call operator [] on the module_ class with some definitions (that's the scope parameter), but the module_ class knows in which Lua state to operate. The scope class is also interesting to look at (some parts are omitted and some slightly simplified):

struct LUABIND_API scope
{
    //...
    explicit scope(detail::registration* reg);
    scope& operator,(scope s);
    void register_(lua_State* L) const;
private:
    detail::registration* m_chain;
};

scope is building a linked list of detail::registration nodes, that list comes from using operator,. So when one does module(L) [class_<...>..., class_<...>...], class_ which inherits from scope initializes its base with a detail::registration instance, then the comma operator of scope builds a linked list of all registrations, this is passed to module_::operator[] which calls scope::register_ which in turn enumerates the chain and calls register_ on all those detail::registration objects. The lua_State is always passed to register_.

Phew. Now let's see what happens when one does class_<C>("C").def("f", &C::f). This constructs an instance of class_<C> with a certain name which goes in the detail::registration member in class_. Calling the class_::def method writes in the reg structure and whatnot, but here's a very interesting line deeper in the call chain from def:

            object fn = make_function(
                L, f, deduce_signature(f, (Class*)0), policies);

Oooh, deduce_signature, I really wanted to see that. Now I want to unsee it, but the way it works is this: through dark preprocessor sorcery aided by boost (BOOST_PP_ITERATE and some other utilities) the following is generated for each N between one and LUABIND_MAX_ARITY:

template <class R, class T, class A1, classA2, ..., classAN>
boost::mpl::vectorN_PLUS_2<R, T, A1, A2, ..., AN>  // type of return value
     deduce_signature(R(T::*)(A1, A2, ..., AN))
     {
          return boost::mpl::vectorN_PLUS_2<R, T, A1, A2, ..., AN>()
     }

Again, a function like this is generated for all N between 1 and LUABIND_MAX_ARITY which is 10 by default. There are a couple of overloads to handle const methods, virtual wrappers and free functions and such, which means that there are around 50 deduce_signature functions that end up in your sources just after the preprocessor and before compilation has started. From there, it's compiler's job to choose the right deduce_signature overload for the functions you pass to def and that will return the correct boost::mpl::vectorX type. From there make_function can do anything - it has a [compile time] list of parameter types and through some more template magic these are counted, converted to and from Lua values and so on.

This is where I will stop. Investigation is based on Luabind 0.8.1. Feel free to browse/debug Luabind's code for more answers - it takes some time but it is not that hard after you get used to the style :) Good luck.

TL;DR: Magic... black magic

Answer 2

luabind has templated wrapper functions for the familiar int luafunction(lua_State* L) prototype which the C API accepts. In essence, the lua_CFunction is created for you. The actual C or C++ function to call can be stored as an upvalue to the wrapper. In the case of a C++ member function, the this pointer can be taken from the first argument.

Example code wrapping a C function using upvalues:

template<typename R, typename T1>
int arg1wrapper(lua_State* L)
{
    typedef R (*F)(T1);
    F func = (F)lua_touserdata(L, lua_upvalueindex(1));
    R retValue = func(luaToC<T1>(L, 1));
    push(L, retValue);
    return 1;
}

// example use
template<typename R, typename T1>
void push(R (*func)(T1))
{
    lua_pushlightuserdata(L, func);
    lua_pushcclosure(L, &arg1wrapper<R, T1>, 1);
}

(The luaToC templated function would be specialized for every C and C++ type the library intends to support. The push function would be overloaded similarily.)

You will notice that the above pair of functions will work for only one particular kind of C function; functions with a non-void return value and a single parameter. Void returns can be easily handled by factoring the return value operations into a third template specialized for void, but to support other amounts of parameters, you need a bunch of overloads. luabind does this: it has one overload for every amount of parameters it supports, including one for 0 parameters (the maximum amount is some arbitrary number they chose).

(note that in C++0x you can use variadic templates to support any amount of parameters with the same template)