Could someone explain __declspec(naked) please?

Question

I'm looking into porting a script engine written for Windows to Linux; it's for Winamp's visualization platform AVS. I'm not sure if it's even possible at the moment. From what I can tell the code is taking the addresses of the C functions nseel_asm_atan and nseel_asm_atan_end and storing them inside a table that it can reference during code execution.

I've looked at MS's documentation, but I'm unsure what __declspec(naked) really does. What is prolog and epilog code mentioned in the documentation? Is that related to Windows calling conventions? Is this portable? Know of any Linux-based examples using similar techniques?

static double (*__atan)(double) = &atan;
__declspec ( naked ) void nseel_asm_atan(void)
{
  FUNC1_ENTER

  *__nextBlock = __atan(*parm_a);

  FUNC_LEAVE
}
__declspec ( naked ) void nseel_asm_atan_end(void) {}

Answer 1

Basically the function prologue sets up a stack frame for local variables and the epilogue takes care of cleaning it up. This is usually done automatically by the compiler. If you use __declspec(naked), setting up this stack frame will be up to you so it gives you more flexibility.

There are many references: here , here, also here and more.

The GNU gcc compiler also supports naked, but apparently not for x86: search for "naked" in the page (I haven't tried to see if it works on x86)

Answer 2

from wikipedia function prologue and epilogue :

In assembly language programming, the function prologue is a few lines of code at the beginning of a function, which prepare the stack and registers for use within the function. Similarly, the function epilogue appears at the end of the function, and restores the stack and registers to the state they were in before the function was called.

To ensure that the compiler does not auto generate an extra code within the your function , always declare the function using the __declspec(naked) convention.

let's take a look at this function :

void myTrampoline()
{
__asm {
  PUSHFD
  PUSHAD
  CALL jumpHookCallback
  POPAD
  POPFD
  POP EAX
  MOV AL, 1
  POP EDI
  POP ESI
  JMP [restoreJumpHook]
 }
}

now the compiler will generate code that manipulate the stack frame of the function called as prologue and epilogue of function and the result will look like this

;Prologue
push ebp
mov ebp, esp
sub esp, N

PUSHFD
PUSHAD
CALL jumpHookCallback
POPAD
POPFD
POP EAX
MOV AL, 1
POP EDI
POP ESI
JMP [restoreJumpHook]

;Epilogue
mov esp, ebp
pop ebp
ret

but if we use __declspec(naked) there will be no Prologue no Epilogue

void __declspec(naked) myTrampoline()
{
__asm {
  PUSHFD
  PUSHAD
  CALL jumpHookCallback
  POPAD
  POPFD
  POP EAX
  MOV AL, 1
  POP EDI
  POP ESI
  JMP [restoreJumpHook]
 }
}

and the result will look like this :

    PUSHFD
    PUSHAD
    CALL jumpHookCallback
    POPAD
    POPFD
    POP EAX
    MOV AL, 1
    POP EDI
    POP ESI
    JMP [restoreJumpHook]