64 bit assembly on Mac OS X runtime errors: "dyld: no writable segment" and "Trace/BPT trap"

Question

When attempting to run the following assembly program:

.globl start

start:
    pushq $0x0 
    movq $0x1, %rax
    subq $0x8, %rsp
    int $0x80

I am receiving the following errors:

dyld: no writable segment
Trace/BPT trap

Any idea what could be causing this? The analogous program in 32 bit assembly runs fine.

Answer 1

OSX now requires your executable to have a writable data segment with content, so it can relocate and link your code dynamically. Dunno why, maybe security reasons, maybe due to the new RIP register. If you put a .data segment in there (with some bogus content), you'll avoid the "no writable segment" error. IMO this is an ld bug.

Regarding the 64-bit syscall, you can do it 2 ways. GCC-style, which uses the _syscall PROCEDURE from libSystem.dylib, or raw. Raw uses the syscall instruction, not the int 0x80 trap. int 0x80 is an illegal instruction in 64-bit.

The "GCC method" will take care of categorizing the syscall for you, so you can use the same 32-bit numbers found in sys/syscall.h. But if you go raw, you'll have to classify what kind of syscall it is by ORing it with a type id. Here is an example of both. Note that the calling convention is different! (this is NASM syntax because gas annoys me)

; assemble with
; nasm -f macho64 -o syscall64.o syscall64.asm && ld -lc -ldylib1.o -e start -o syscall64 syscall64.o
extern _syscall
global start

[section .text align=16]
start:
    ; do it gcc-style
    mov rdi, 0x4 ; sys_write
    mov rsi, 1 ; file descriptor
    mov rdx, hello
    mov rcx, size
    call _syscall ; we're calling a procedure, not trapping.
    
    ;now let's do it raw
    mov rax, 0x2000001 ; SYS_exit = 1 and is type 2 (bsd call)
    mov rdi, 0 ; Exit success = 0
    syscall ; faster than int 0x80, and legal!
    
    
[section .data align=16]
hello: db "hello 64-bit syscall!", 0x0a
size: equ $-hello

check out http://www.opensource.apple.com/source/xnu/xnu-792.13.8/osfmk/mach/i386/syscall_sw.h for more info on how a syscall is typed.

Answer 2

The system call interface is different between 32 and 64 bits. Firstly, int $80 is replaced by syscall and the system call numbers are different. You will need to look up documentation for a 64-bit version of your system call. Here is an example of what a 64-bit program may look like.