Constraining r10 register in gcc inline x86_64 assembly

Question

I'm having a go at writing a very light weight libc replacement library so that I can better understand the kernel - application interface. The first task is clearly getting some system call wrappers in place. I've successfully got 1 to 3 argument wrappers working but I'm struggling with a 4 argument varient. Here's my starting point:

long _syscall4(long type, long a1, long a2, long a3, long a4)
{
    long ret;
    asm
    (
        "syscall"
        : "=a"(ret)    // return value
        : "a"(type), "D"(a1), "S"(a2), "d"(a3), "r10"(a4)
        : "c", "r11", "memory"  // some syscalls read or write memory
                   // the syscall insn instruction itself destroys RCX and R11
    );
    return ret;
}

(Editor's note: this is safe and usable, and a good example, after applying answer's way to handle r10. MUSL libc has some similar macros.)

The compiler gives me the following error:

error: matching constraint references invalid operand number

My _syscall3 function works fine but doesn't use r10 or have a clobber list.

(Editor's note: it wouldn't be safe to have no clobber list: you need to tell the compiler that RCX and R11 are overwritten, and that "memory" should be in sync before the system call which may read or write memory. If you wanted to write specific wrappers for specific system calls, you could selectively omit "memory" clobbers, or use dummy memory operands based on which parameters are pointers for that system call.

If this _syscall4 function can't inline, the register and "memory" clobbers won't in practice cause any problem, but you should make these able to inline; inlining this system call will take less code at the call site than calling a non-inline wrapper function.)

Answer 1

There are no constraints for registers: %r8 .. %15. However, more recent (as in gcc-4.x) should accept:

register long r10 asm("r10") = a4;

then use the input constraint: "r" (r10) for your asm statement.
https://gcc.gnu.org/onlinedocs/gcc/Local-Register-Variables.html

---

Note that forcing the choice of an "r" constraint for Extended asm is the only behaviour that GCC guarantees for register-asm locals. Things like register void *rsp asm("rsp"); and void *stack_pointer = rsp; do sometimes work, but are not guaranteed and not recommended anymore.

---

You're going to want your syscall wrapper asm statement to be volatile and have a "memory" clobber, unless you write specific wrappers for specific system calls to know which args are pointers, using a dummy memory input or output (as per How can I indicate that the memory *pointed* to by an inline ASM argument may be used?)

It needs to volatile because doing write(1, buf, 16) should print the buffer twice, not just CSE the return value! System calls are in general not Pure functions of their inputs, so you need volatile.

(Some specific system call wrappers like getpid could be non-volatile, because they do return the same thing every time, unless you also use fork. But getpid is more efficient if done through the VDSO so it doesn't have to enter the kernel in the first place if you're on Linux, so if you're making a custom wrapper for getpid and clock_gettime you probably don't want syscall in the first place. See The Definitive Guide to Linux System Calls)

The "memory" clobber is needed because a pointer in a register does not imply that the pointed-to memory is also an input or output. Stores to a buffer that are only read by a write system call need to not be optimized away as dead stores. Or for munmap, the compiler had better have finished any loads/stores before the memory is unmapped. Some system calls don't take any pointer inputs, and don't need "memory", but a generic wrapper has to make worst-case assumptions.

register ... asm("r10") does not in general require asm volatile or "memory" clobbers, but a syscall wrapper does.