Why does ignoring SIGTRAP not work with asm?

Question

I'm trying to ignore SIGTRAP. I have the following proof-of-concept code:

#include <signal.h>
#include <stdlib.h>
int main(){
    signal(SIGTRAP, SIG_IGN);
    write(1, "A", 1);
    asm("int3");
    write(1, "B", 1);
    return 0;
}

When I run it, I expect to see "AB", but I see

ATrace/breakpoint trap (core dumped)

Why does my program terminate despite it ignoring SIGTRAP?

Answer 1

According to this site a blocked/ignored signal is automatically unblocked inside the kernel code when it is raised. So if the same signal is raised repeatedly, an infinite loop will not happen. Instead the application is terminated on the second signal raise, at least in the Linux kernel implementation.

So when using raise(), the SIGTRAP will only be raised once, causing no problems. But with asm("int3") the processor will re-execute the instruction which raised the signal. The second time around this causes process termination.

The relevant kernel source (for the old 2.6.27) is here (function force_sig_info):

939        if (blocked || ignored) {
940                action->sa.sa_handler = SIG_DFL;
941                if (blocked) {
942                        sigdelset(&t->blocked, sig);
943                        recalc_sigpending_and_wake(t);
944                }
945        }