Segmentation fault creating a user-level thread with C and assembly

Question

I am trying to understand some OS fundamentals using some assignments. I have already posted a similar question and got satisfying answers. But this one is slightly different but I haven't been able to debug it. So here's what I do:

What I want to do is to start a main program, malloc a space, use it as a stack to start a user-level thread. My problem is with return address. Here's the code so far:

[I'm editing my code to make it up-to-date to the current state of my answer ]

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>

#define STACK_SIZE 512

void switch_thread(int*,int*);

int k = 0;

void simple_function()
{
    printf("I am the function! k is: %d\n",k);
    exit(0);
}

void create_thread(void (*function)())
{   
   int* stack = malloc(STACK_SIZE + 32);
   stack = (int* )(((long)stack & (-1 << 4)) + 0x10);
   stack = (int* ) ((long)stack + STACK_SIZE); 
   *stack = (long) function;
   switch_thread(stack,stack);  
}

int main()
{
    create_thread(simple_function);
    assert(0);
    return 0;
}

switch_thread is an assembly code I've written as follows:

.text
    .globl  switch_thread
switch_thread:  
    movq    %rdi, %rsp
    movq    %rsi, %rbp
    ret

This code runs really well under GDB and gives the expected output (which is,passing the control to simple_function and printing "I am the function! k is: 0". But when run separately, this gives a segmentation fault. I'm baffled by this result.

Any help would be appreciated. Thanks in advance.

Answer 1

Two problems with your code:

1. Unless your thread is actually inside a proper procedure (or a nested procedure), there's no such thing as "base pointer". This makes the value of %rbp irrelevant since the thread is not inside a particular procedure at the point of initialization.

2. Contrary to what you think, when the ret instruction gets executed, the value that %rsp is referring to becomes the new value of the program counter. This means that instead of *(base_pointer + 1), *(base_pointer) will be consulted when it gets executed. Again, the value of %rbp is irrelevant here.

Your code (with minimal modification to make it run) should look like this:

void switch_thread(int* stack_pointer,int* entry_point);

void create_thread(void (*function)())
{
    int* stack_pointer = malloc(STACK_SIZE + 8);
    stack_pointer += STACK_SIZE; //you'd probably want to back up the original allocated address if you intend to free it later for any reason.
    switch_thread(stack_pointer,function);      
}

Your switch_thread routine should look like this:

    .text
    .globl  switch_thread
switch_thread:
    mov     %rsp, %rax //move the original stack pointer to a scratch register
    mov     %rdi, %rsp //set stack pointer
    push    %rax       //back-up the original stack pointer
    call    %rsi       //call the function
    pop     %rsp       //restore the original stack pointer
    ret                //return to create_thread

FYI: If you're initializing a thread on your own, I suggest that you first create a proper trampoline that acts as a thread entry point (e.g. ntdll's RtlUserThreadStart). This will make things much cleaner especially if you want to make your program multithreaded and also pass in any parameters to the start routine.