Get line number of segfault using signal handler [duplicate]

Question

I'm running a program on a remote machine that I can only interact with using stdout. I have a segfault somewhere in the program and I'm trying to figure out where. Is it possible to write a signal handler for sigsegv that gives me the line number and the file where it happened?

Answer 1

That sounds like a horrible debugging environment, but if it's really infeasible to get GDB running, you could try the following (assuming Linux):

1. Register a handler for SIGSEGV, SIGBUS, or whatever you think the fatal signal is using sigaction(), and pass sa.sa_flags = SA_SIGINFO. Use the sa_sigaction rather than sa_handler member of struct sigaction to register the handler.
2. The handler will receive a void *context argument. Assuming X86_64 (you will have to figure out what the corresponding index is for the instruction pointer on other architectures), you can get the address where the signal was triggered via ((ucontext_t*)context)->uc_mcontext.gregs[REG_RIP].
3. Once you have the address, run e.g. addr2line -Cfip -e <binary with debugging symbols> <address> to get the function and line number. (If you're cross-compiling, you need to use the addr2line from the toolchain, which will probably have some prefix, e.g. arm-linux-androideabi-addr2line.)

(Incidentally, I just recalled the context signal handler argument from another question. :)

A limitation to the above approach is that it probably won't give you a line number for crashes inside libraries -- especially if they're loaded at random addresses.

Another approach is to use backtrace(3), which is available in glibc and a few other libc's. Bit hackish, but you could write the addresses you get from it (as strings) to popen("addr2line -Cfip -e <binary with debugging symbols>") (/proc/self/exe could be used too on Linux) to generate a backtrace with line numbers on stdout. backtrace_symbols_fd() is worth looking into too, though it won't give you line numbers. It needs -rdynamic when compiling.

Edit:

It seems GDB uses personality(2) and ADDR_NO_RANDOMIZE to turn of address space randomization for libraries (would prolly need to be followed by a re-execve()). If you're really desperate, maybe that (or /proc/sys/kernel/randomize_va_space) could be used to get line numbers inside libraries too.