Are the prologue and epilogue mandatory when writing assembly functions?

Question

Recently I rewrote some libc functions in assembly and for some of them (the one that didn't need any call or syscall instructions, like strlen), I discarded the prologue and epilogue because my tests didn't failed without it (maybe they were not complex enough). During peer review, someone told me it was bad practice to discard them, but couldn't explain me why.

So, am I running into problems when I call asm functions that don't have the prologue/epilogue combination?

Is it a good practice to add it even when no additionnal space is needed on the stack?

If mandatory for some reasons, why doesn't the assembler (I used nasm) take care of it?

Answer 1

If you do not set up a proper stack frame, it can be hard for a debugger to know what function you are in right now. On ELF-targets, you have to manually provide CFI data (cf. this article) if you do not explicitly set up a stack frame. Without CFI data, stack unwinding doesn't work and the debugger might not be able to find out what function you are in. Unless you want to manually add CFI data (which is somewhat tedious and easy to get wrong), I recommend you to accept the minor performance loss and just set up a full stack frame.

Answer 2

Are the prologue and epilogue mandatory when writing assembly functions?

For pure assembly you don't even need to have "functions" - e.g. you can have a piece of code with multiple different entry points and a single "ret" (which is equivalent to what you might end up with after a decent compiler does "tail call" optimizations).

For writing functions that are compatible with someone else's calling conventions; you have to comply with someone else's calling conventions. If those calling conventions say that (e.g.) some registers must be preserved by callee, then the callee has to preserve those registers (by saving in prologue and loading in epilogue) and if it doesn't you can end up with unexpected data corruption (because compiler expected a value to remain unchanged but..).

Note that for 80x86, none of the calling conventions require a stack frame (as EBP or RBP) - that's just historical memorabilia resulting from the poor design of ancient debuggers, and ceased being sane when debuggers switched to better techniques about 20 years ago.

If mandatory for some reasons, why doesn't the assembler (I used nasm) take care of it?

Assemblers typically have no idea which calling convention (if any) you're trying to comply with.