Hello World using NASM (and GoLink) on x64 64 bit Windows

Question

I am currently learning assembly language for a custom compiler project. However, i wasn't able to make a single working assembly program. I have an AMD CPU with a 64 bit operating system and a x64 based processor. I am using Windows 11.

I don't want to use c libraries as it kind of destroys the purpose of coding as low level as possible which is my objective.

I already tried a lot of different programs and linkers.

Assembly Syscalls in 64-bit Windows This doesn't contain any working code.

Hello world using nasm in windows assembly I think this is for 32 bit systems, but i don't know what i need to change. I changed "eax" to "rax" and this compiled and run but didn't print anything. I was also able to add random letters to thecode without any errors and still an exit code of 0. (Just one GoLink warning)

Setting Exit Status in NASM for Windows Same problem here. I always get an exit code of one and the compiler doesn't care about any syntax errors.

I also tried using gcc and ld without success.

What am i doing wrong?

Edit I tried this: How to write hello world in assembly under Windows? but link.exe seems to be only for visual studio (i am using vscode). When i use GoLink, i get the following error:

Error!
The following symbol was not defined in the object file or files:-
MessageBoxA
Output file not made

Edit 2 When im using gcc, i get the following error:

C:/msys64/ucrt64/bin/../lib/gcc/x86_64-w64- 
mingw32/13.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: 
out.obj:out.asm:(.text+0x1e): undefined reference to `MessageBoxA'
collect2.exe: error: ld returned 1 exit status

If i remove the message printing code, it works fine and i finally get an exit code of my choice! Is the MessageBoxA function in another library or something? How can i fix this?

Edit 3 I found this code and copied it, but again, it doesn't work. (editor's note: it's 32-bit code for the 32-bit calling convention.)

global _main
    extern  GetStdHandle
    extern  WriteFile
    extern  ExitProcess

    section .text
    _main:
        ; DWORD  bytes;    
        mov     ebp, esp
        sub     esp, 4

        ; hStdOut = GetstdHandle( STD_OUTPUT_HANDLE)
        push    -11
        call    GetStdHandle
        mov     ebx, eax    

        ; WriteFile( hstdOut, message, length(message), &bytes, 0);
        push    0
        lea     eax, [ebp-4]
        push    rax
        push    (message_end - message)
        push    message ; ERROR
        push    rbx
        call    WriteFile

        ; ExitProcess(0)
        xor rcx, rcx
        call    ExitProcess

        ; never here
        hlt
    message: db      "Hello, World", 8
    message_end:

When i run this with the following command:

nasm -f win64 out.asm && gcc -o  out.exe out.obj -nostdlib -lkernel32

... i get a truncation error (idk what that means)

out.obj:out.asm:(.text+0x18): relocation truncated to fit: 
IMAGE_REL_AMD64_ADDR32 against `.text'
collect2.exe: error: ld returned 1 exit status

I am a bit irritated why it's so hard to execut such a simple program. I just need a hello world program for nasm, x64_64 windows with visual studio code and without c librarys.

Answer 1

After quite a bit of trying, I finally got it to work. Here is a cleaned-up version (by @PeterCordes) of the code that was in the first version of this answer. (Again, I am using Windows with an x64-based processor from AMD. I am coding in Visual Studio Code.)

default rel

extern GetStdHandle
extern WriteFile
extern ExitProcess

section .text
global main
main:
    sub     rsp, 40          ; reserve shadow space and align stack by 16

    mov     rcx, -11         
    call    GetStdHandle

    mov     rcx, rax         ; HANDLE is a 64-byte type on x64
    lea     rdx, [msg]       ; lpBuffer = RIP-relative LEA (default rel)
    mov     r8d, msg.len     ; DWORD nNumberOfBytesToWrite = 32-bit immediate constant length
    lea     r9, [rsp+48]     ; lpNumberOfBytesWritten pointing into main's shadow space
    mov     qword [rsp + 32], 0   ; lpOverlapped = NULL; This is a pointer, needs to be qword.
    call    WriteFile        ; WriteFile(handle, msg, len, &our_shadow_space, NULL)

;;; BOOL return value in EAX (BOOL is a 4-byte type, unlike bool).
;;; NumberOfBytesWritten in dword [rsp+48]

    xor     ecx, ecx
    call ExitProcess
  ; add     rsp, 40      ; alternative to calling a noreturn function like ExitProcess
  ; ret


section .data         ; or section .rdata to put it in a read-only page
    msg:  db "Hello, World!", 13, 10    ; including a CR LF newline is a good idea for a line of output text
    .len equ  $-msg    ; let the assembler calculate the string length
                       ; .len is a local label that appends to the most recent non-dot label, so this is msg.len

Assemble and link it with nasm and gcc:

nasm -f win64 out.asm   &&  gcc -o  out.exe out.obj -nostdlib -lkernel32

Omitted from this example: stack-unwind metadata for instructions that change RSP (sub rsp,40). It will run fine without it as long as no SEH or C++ exceptions happen, but debugger backtracing up from inside WriteFile back through main to its caller won't work. See Under what conditions do I need to set up SEH unwind info for an x86-64 assembly function? for more. If you're a beginner in asm and just want to understand how the "normal" stuff works, and aren't planning to actually use hand-written asm in production code, you don't need to worry about SEH and stack unwind info at all.

---

Commentary on the differences between this and the code in How To Properly call 64 Bit Windows API In Assembly it was originally based on:

The only data we want in static storage (.data / .bss / .rdata) are the ASCII bytes of the string itself. The output number of bytes can be pointing to stack space, and the constant we pass to WriteFile can just be an immediate in the machine code, not loaded from static storage (dq 14). Also, it's generally good to have the assembler calculate a length instead of hard-coding it.

WriteFile needs somewhere to store the number of bytes actually written (since the return value is just a BOOL, unfortunately, unlike POSIX ssize_t write() which returns negative for error.) We can use stack space (like a local variable) for this. Since main is a function, it has its own shadow space.

(The example code copied from How To Properly call 64 Bit Windows API In Assembly used weird names like NtlpNBytesWritten for a variable in BSS to point the lpNumberOfBytesWritten output arg of WriteFile at. That's confusing since these aren't low-level NT functions, they're the Win32 API. And the variable itself is just a dword so shouldn't have lp in its name if you are using static storage.)

The arg names in comments are from MS's documentation for WriteFile: https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-writefile

---

Other Hello World examples exist, such as an answer on How to write hello world in assembly under Windows? - that uses MessageBoxA, which is in user32.dll, vs. GetStdHandle and WriteFile being in kernel32.dll