Under what conditions do I need to set up SEH unwind info for an x86-64 assembly function?

Question

The 64-bit Windows ABI defines a generalized exception handling mechanism, which I believe is shared across C++ exceptions and structured exceptions available even in other languages such as C.

If I'm writing an x86-64 assembly routine to be compiled in nasm and linked into a C or C++ library, what accommodations do I need make on Windows in terms of generating unwind info and so on?

I'm not planning on generating any exceptions directly in the assembly code, although I suppose it is possible that the code may get an access violation if a user-supplied buffer is invalid, etc.

I'd like the write the minimum possible to get this to work, especially since it seems that nasm has poor support for generating unwind info and using MASM is not an option for this cross-platform project. I do need to use (hence save and restore) non-volatile registers.

Answer 1

As a general rule, Windows x64 requires all functions to provide unwind information. The only exception is for leaf functions which do not modify rsp and do not modify any nonvolatile registers.

Answer 2

Judging by the context of your question, what you really want to know is the practical consequences of not providing unwind information for your non-leaf assembly functions on x64 Windows. Since C++ exceptions are implemented based on SEH exceptions, when I talk about exceptions below, I mean both all "native" (access violation, something thrown using RaiseException, etc.) and C++ exceptions. Here's a list off the top of my head:

• Exceptions won't be able to pass through your function

It's important to note that this point is not about throwing an exception, or an access violation happening directly in your function. Let's say your assembly code calls into a C++ function, which throws an exception. Even if the caller of your assembly function has a matching catch block, it will never be able to catch the exception, as unwinding will stop at your function without the unwind data.

• When walking the stack, the stack walk will stop at the function without unwind data (or go astray; the point is, you will get an invalid call stack)

Basicaly, anything that walks the stack is screwed if your function is present on the call stack (debuggers when displaying the call stack, profilers, etc.)

• Registered Unhandled Exception Filters will not be called back if an exception gets thrown, and your assembly function is on the call stack

This interferes with anything that relies on UEFs. Custom crash handlers, for instance. Or something potentially more relevant: std::terminate won't be called back in this case, if your program throws a C++ exception, that is unhandled (as it's dictated by the C++ standard). The MSVC runtime uses a UEF to implement this, so this won't work as well.

---

Are you developing a 3rd party library? If that's the case, the importance of the above points will depend on the use case of your clients.