Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Only question marks in backtrace reported by gdb on ARM

I'm trying to debug a software with gdbserver on ARM to get a backtrace of a crash. Unfortunately I get only question marks. Everywhere, I read this problem is simply related to the lack of symbols, but symbols are not stripped from my libraries.

If I try to use the file command to load the symbols in the client I get:

reading symbols from <path>/libQtWebKit.so.4.7.2...(no debugging symbols found)...done.

and then, when the crash occurs:

Program received signal SIGSEGV, Segmentation fault.
0x00000000 in ?? ()
(gdb) bt
#0  0x00000000 in ?? ()
#1  0x4bf38b88 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

My libraries are compiled in release but the symbols are actually there. With nm I can find those. Why do I only get question marks? Is this only because the libraries are compiled with optimization? Isn't it possible to debug with libraries in release mode?

like image 303
Luca Carlon Avatar asked Dec 08 '11 20:12

Luca Carlon


People also ask

What does backtrace in GDB do?

A backtrace is a summary of how your program got where it is. It shows one line per frame, for many frames, starting with the currently executing frame (frame zero), followed by its caller (frame one), and on up the stack. Print a backtrace of the entire stack: one line per frame for all frames in the stack.

How do you do a full backtrace?

To print a backtrace of the entire stack, use the backtrace command, or its alias bt . This command will print one line per frame for frames in the stack. By default, all stack frames are printed. You can stop the backtrace at any time by typing the system interrupt character, normally Ctrl-c .

How do you find the backtrace?

A backtrace is a list of the function calls that are currently active in a thread. The usual way to inspect a backtrace of a program is to use an external debugger such as gdb. However, sometimes it is useful to obtain a backtrace programmatically from within a program, e.g., for the purposes of logging or diagnostics.


2 Answers

The corrupt stack note is probably your problem. It looks like a return address or virtual table entry or something was overwritten with zeros, and then control was transferred there. Even if you have symbols available, those addresses aren't pointing to valid symbols. Hence the segfault.

I don't envy your task. These are some of the hardest bugs to track down, and can even move or temporarily go away when you make code changes to try and catch them. Your best bet is usually something like git bisect or your VCS equivalent to find the commit that introduced it. Hopefully it isn't too difficult to reproduce.

like image 59
Karl Bielefeldt Avatar answered Sep 22 '22 08:09

Karl Bielefeldt


One trick you can sometimes use when you get the "SEGV at address 0" problem is to manually pop the return address from the top of the stack into the pc and trying to do a stack trace from there. This assumes that you got to address 0 by doing an indirect call through a NULL pointer, which is the most common way of getting to address 0.

Now I'm not too familiar with ARM, but on an x86 PC, you would do:

(gdb) set $eip = *(void **)$esp
(gdb) set $esp = $esp + 4

and then do another backtrace to figure out where you really are.

If you can figure out the calling convention used for ARM by your compiler, you should be able to do something similar.

like image 20
Chris Dodd Avatar answered Sep 22 '22 08:09

Chris Dodd