How can I expose C functions to a custom virtual machine?

Question

I'm working on a virtual machine which I would like to be able to interface with C. Going the other way and exposing virtual machine functions to C code is fairly easy, what I can't wrap my head around is exposing C functions to a virtual machine.

I'd like to be able to dynamically register C functions with the virtual machine like so:

vm_register(printf);

Then in my virtual machine, push the arguments to the stack, and:

call printf

The problem is that without knowing how many arguments the function requires, and of what type, I'm not sure function pointers can be used.

Is there a generic function pointer type that can be used in this situation? Can someone steer me in the right direction?

Answer 1

The general answer is that you have to implement it yourself using assembly. After linking with libc, you have the address of the function you want to call, and you have to pass the parameters to the function manually (using the calling convention of whatever platform your virtual machine is running on).

Luckily there's a library, libffi, that does exactly what you want. It's pretty easy to use as well, its source includes some documentation and examples. If you're interested to see how it works, you can take a look at its code (e.g. calling a function using the unix calling convention).

Regarding the parameter types, you usually have to let the user describe them for you and blindly accept that and pass them further on to libffi (or to the hardware if you do it without libffi). Another way would be to parse the C header file for the function to call, which is less error prone - but in any case, there's really no safe way as the binary code for the function doesn't describe its interface (with printf and its variable parameter list even more so).