Using interrupt 0x80 on 64-bit Linux [duplicate]

Question

I have a simple 64-bit assembly program which is intended to print an 'O' and 'K' followed by a newline.

However, the 'K' is never printed. One of the goals of the programs is to print the value in the lower bits of the rax register as ASCII letter. The program is specifically for 64-bit Linux, written for educational purposes, so there is no need to use C-style system calls.

I suspect that the problem either lies with mov QWORD [rsp], rax or mov rcx, rsp.

Currently, the program only outputs 'O' followed by a newline.

How can one change the program to make it use the value in rax and then print a 'K' so that the complete output is 'OK' followed by a newline?

bits 64

section .data

o:  db "O"      ; 'O'
nl: dq 10       ; newline

section .text

;--- function main ---
global main         ; make label available to the linker
global _start       ; make label available to the linker
_start:             ; starting point of the program
main:               ; name of the function

;--- call interrupt 0x80 ---
mov rax, 4          ; function call: 4
mov rbx, 1          ; parameter #1 is 1
mov rcx, o          ; parameter #2 is &o
mov rdx, 1          ; parameter #3 is length of string
int 0x80            ; perform the call

;--- rax = 'K' ---
mov rax, 75         ; rax = 75

;--- call interrupt 0x80 ---
sub rsp, 8          ; make some space for storing rax on the stack
mov QWORD [rsp], rax        ; move rax to a memory location on the stack
mov rax, 4          ; function call: 4
mov rbx, 1          ; parameter #1 is 1
mov rcx, rsp            ; parameter #2 is rsp
mov rdx, 1          ; parameter #3 is length of string
int 0x80            ; perform the call
add rsp, 8          ; move the stack pointer back

;--- call interrupt 0x80 ---
mov rax, 4          ; function call: 4
mov rbx, 1          ; parameter #1 is 1
mov rcx, nl         ; parameter #2 is nl
mov rdx, 1          ; parameter #3 is length of string
int 0x80            ; perform the call

;--- exit program ---
mov rax, 1          ; function call: 1
xor rbx, rbx            ; return code 0
int 0x80            ; exit program

Update: Note that this is a 64-bit x86 Assembly program that uses int 80h, and is very different from a 32-bit x86 Assembly program that uses int 80h.

Answer 1

Obviously you write a 64-bit program and you use the "int 0x80" instruction. "int 0x80" however only works correctly in 32-bit programs.

The address of the stack is in a range that cannot be accessed by 32-bit programs. Therefore it is quite probable that "int 0x80"-style system calls do not allow accessing this memory area.

To solve this problem there are two possibilities:

• Compile as 32-bit application (use 32-bit registers like EAX instead of 64-bit registers like RAX). When you link without using any shared libraries 32-bit programs will work perfectly on 64-bit Linux.
• Use "syscall"-style system calls instead of "int 0x80"-style system calls. The use of these differs a lot from "int 0x80"-style ones!

32-bit code:

mov eax,4    ; In "int 0x80" style 4 means: write
mov ebx,1    ; ... and the first arg. is stored in ebx
mov ecx,esp  ; ... and the second arg. is stored in ecx
mov edx,1    ; ... and the third arg. is stored in edx
int 0x80

64-bit code:

mov rax,1    ; In "syscall" style 1 means: write
mov rdi,1    ; ... and the first arg. is stored in rdi (not rbx)
mov rsi,rsp  ; ... and the second arg. is stored in rsi (not rcx)
mov rdx,1    ; ... and the third arg. is stored in rdx
syscall

--- Edit ---

Background information:

"int 0x80" is intended for 32-bit programs. When called from a 64-bit program it behaves the same way it would behave like if it has been called from a 32-bit program (using the 32-bit calling convention).

This also means that the parameters for "int 0x80" will be passed in 32-bit registers and the upper 32 bits of the 64-bit registers are ignored.

(I just tested that on Ubuntu 16.10, 64 bit.)

This however means that you can only access memory below 2^32 (or even below 2^31) when using "int 0x80" because you cannot pass an address above 2^32 in a 32-bit register.

If the data to be written is located at an address below 2^31 you may use "int 0x80" to write the data. If it is located above 2^32 you can't. The stack (RSP) is very likely located above 2^32 so you cannot write data on the stack using "int 0x80".

Because it is very likely that your program will use memory above 2^32 I have written: "int 0x80 does not work with 64-bit programs."