Linux syscall, libc, VDSO and implementation dissection

Question

I dissects the syscall call in the last libc:

git clone git://sourceware.org/git/glibc.git

And I have this code in sysdeps/unix/sysv/linux/i386/sysdep.h:

#   define INTERNAL_SYSCALL_MAIN_INLINE(name, err, nr, args...) \
LOADREGS_##nr(args)                         \
asm volatile (                          \
"call *%%gs:%P2"                            \
: "=a" (resultvar)                          \
: "a" (__NR_##name), "i" (offsetof (tcbhead_t, sysinfo))        \
  ASMARGS_##nr(args) : "memory", "cc")

If I understand well this code, the LOADREGS_##nr(args) macro loads the argument in the registers ebx, ecx, edx, esi, edx and ebp.

sysdeps/unix/sysv/linux/i386/sysdep.h

# define LOADREGS_0()
# define ASMARGS_0()
# define LOADREGS_1(arg1) \
    LOADREGS_0 ()
# define ASMARGS_1(arg1) \
    ASMARGS_0 (), "b" ((unsigned int) (arg1))
# define LOADREGS_2(arg1, arg2) \
    LOADREGS_1 (arg1)
# define ASMARGS_2(arg1, arg2) \
    ASMARGS_1 (arg1), "c" ((unsigned int) (arg2))
# define LOADREGS_3(arg1, arg2, arg3) \
    LOADREGS_2 (arg1, arg2)
# define ASMARGS_3(arg1, arg2, arg3) \
    ASMARGS_2 (arg1, arg2), "d" ((unsigned int) (arg3))
# define LOADREGS_4(arg1, arg2, arg3, arg4) \
    LOADREGS_3 (arg1, arg2, arg3)
# define ASMARGS_4(arg1, arg2, arg3, arg4) \
    ASMARGS_3 (arg1, arg2, arg3), "S" ((unsigned int) (arg4))
# define LOADREGS_5(arg1, arg2, arg3, arg4, arg5) \
    LOADREGS_4 (arg1, arg2, arg3, arg4)
# define ASMARGS_5(arg1, arg2, arg3, arg4, arg5) \
    ASMARGS_4 (arg1, arg2, arg3, arg4), "D" ((unsigned int) (arg5))
# define LOADREGS_6(arg1, arg2, arg3, arg4, arg5, arg6) \
    register unsigned int _a6 asm ("ebp") = (unsigned int) (arg6); \
    LOADREGS_5 (arg1, arg2, arg3, arg4, arg5)
# define ASMARGS_6(arg1, arg2, arg3, arg4, arg5, arg6) \
    ASMARGS_5 (arg1, arg2, arg3, arg4, arg5), "r" (_a6)
#endif /* GCC 5  */
    enter code here

Where is the code which load the argument in the registers ebx, ecx, edx, esi, edx and ebp? it's this code above? I don't understand the implementation. the following code load the 6th argument in the ebx register?

register unsigned int _a6 asm ("ebp") = (unsigned int) (arg6);

What does this code:

ASMARGS_0 (), "b" ((unsigned int) (arg1))

It loads the first argument in the ebx register?

Then the "call *%%gs:%P2" jump to the VDSO code ? this code correspond to "call *gs:0x10"?

so, this following diagram for the write syscall, it's good?:

write(1, "A", 1)  ----->   LIBC   ----->   VDSO   -----> KERNEL
                          load reg           ?   
                        jump to vdso 
|---------------------------------------------------|--------------|
       user land                                       kernel land

I doesn't understand the VDSO utility! the vdso choose the syscall method (sysenter or int 0x80).

Thank's you in advance for your help. And sorry my inglish is very bad.

Answer 1

The macros involved in glibc's syscalls will expand to something like the following, for the example of the exit syscall.

LOADREGS_1(args)
asm volatile (
"call *%%gs:%P2"
: "=a" (resultvar)
: "a" (__NR_exit), "i" (offsetof (tcbhead_t, sysinfo))
  ASMARGS_1(args) : "memory", "cc")

LOADREGS_1(args) will expand to LOADREGS_0(), which will expand to nothing - LOADREGS_*(...) only need to adjust registers when more parameters are provided.

ASMARGS_1(args) will expand to ASMARGS_0 (), "b" ((unsigned int) (arg1)), which will expand to , "b" ((unsigned int) (arg1).

__NR_exit is 1 on x86.

As such, the code will expand to something like:

asm volatile (
"call *%%gs:%P2"
: "=a" (resultvar)
: "a" (1), "i" (offsetof (tcbhead_t, sysinfo))
, "b" ((unsigned int) (arg1) : "memory", "cc")

ASMARGS_* don't actually execute code per se - they're instructions to gcc to make sure that certain values (such as (unsigned int) (arg1)) are in certain registers (such as b, aka ebx). As such, the combination of parameters to asm volatile (which isn't a function, of course, but just a gcc builtin) simply specify how gcc should prepare for the syscall and how it should continue after the syscall completes.

Now, the generated assembly will look something like this:

; set up other registers...
movl $1, %eax
call *%gs:0x10
; tear down

%gs is a segment register that references thread-local storage - specifically, glibc is referencing a saved value that points to the VDSO, which it stored there when it first parsed the ELF headers that told it where the VDSO was at.

Once the code enters the VDSO, we don't know exactly what happens - it varies depending on the kernel version - but we do know that it uses the most efficient available mechanism to run a syscall, such as the sysenter instruction or the int 0x80 instruction.

So, yes, your diagram is accurate:

write(1, "A", 1)  ----->   LIBC   ----->   VDSO   -----> KERNEL
                          load reg           ?   
                        jump to vdso 
|---------------------------------------------------|--------------|
       user land                                       kernel land

Here's a simpler example of code to call into the VDSO, specifically for one-parameter syscalls, from a library that I maintain called libsyscall:

_lsc_syscall1:
    xchgl 8(%esp), %ebx
    movl 4(%esp), %eax
    call *_lsc_vdso_ptr(,1)
    movl 8(%esp), %ebx
    # pass %eax out
    ret

This simply moves parameters from the stack into registers, calls into the VDSO via a pointer loaded from memory, restores the other registers to their previous state, and returns the result of the syscall.