easiest way to debug crash in native library, linked by Android app?

Question

I have ported and created several low-level C-libraries to Android for my use in my application. I cross-compiled them using the NDK, and then link to them using System.loadLibrary(). After some amount of time, my application crashes, seemingly due to a bug in the library:

07-28 11:31:21.675: INFO/DEBUG(2880): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
07-28 11:31:21.675: INFO/DEBUG(2880): Build fingerprint: 'verizon/voles/sholes/sholes:2.2.2/FRG83G/91102:user/release-keys'
07-28 11:31:21.675: INFO/DEBUG(2880): pid: 2893, tid: 2894  >>> com.gnychis.coexisyst <<<
07-28 11:31:21.675: INFO/DEBUG(2880): signal 11 (SIGSEGV), fault addr 2d4eedb4
07-28 11:31:21.675: INFO/DEBUG(2880):  r0 2d4eeda4  r1 00000000  r2 00000024  r3 00000000
07-28 11:31:21.675: INFO/DEBUG(2880):  r4 00d3e770  r5 00000000  r6 4184ff98  r7 4184ffa4
07-28 11:31:21.675: INFO/DEBUG(2880):  r8 100ffad0  r9 4184ff9c  10 4184ff84  fp 100ffe30
07-28 11:31:21.675: INFO/DEBUG(2880):  ip 85b7efec  sp 100ffa88  lr 845d13f8  pc 845f8c38  cpsr 60000010
07-28 11:31:21.675: INFO/DEBUG(2880):  d0  6472656767756265  d1  4472fb3844714069
07-28 11:31:21.675: INFO/DEBUG(2880):  d2  4186eeec4186ee6e  d3  4186ef544186ef74
07-28 11:31:21.675: INFO/DEBUG(2880):  d4  4186ef544186ef20  d5  418998f84186ef88
07-28 11:31:21.675: INFO/DEBUG(2880):  d6  418999604189992c  d7  418999c841899994
07-28 11:31:21.675: INFO/DEBUG(2880):  d8  0000000000000000  d9  0000000000000000
07-28 11:31:21.675: INFO/DEBUG(2880):  d10 0000000000000000  d11 0000000000000000
07-28 11:31:21.675: INFO/DEBUG(2880):  d12 0000000000000000  d13 0000000000000000
07-28 11:31:21.675: INFO/DEBUG(2880):  d14 0000000000000000  d15 0000000000000000
07-28 11:31:21.675: INFO/DEBUG(2880):  d16 0113580000000001  d17 3fe999999999999a
07-28 11:31:21.675: INFO/DEBUG(2880):  d18 42eccefa43de3400  d19 3fe00000000000b4
07-28 11:31:21.675: INFO/DEBUG(2880):  d20 4008000000000000  d21 3fd99a27ad32ddf5
07-28 11:31:21.675: INFO/DEBUG(2880):  d22 3fd24998d6307188  d23 3fcc7288e957b53b
07-28 11:31:21.675: INFO/DEBUG(2880):  d24 3fc74721cad6b0ed  d25 3fc39a09d078c69f
07-28 11:31:21.675: INFO/DEBUG(2880):  d26 0000000000000000  d27 0000000000000000
07-28 11:31:21.675: INFO/DEBUG(2880):  d28 0000000000000000  d29 0000000000000000
07-28 11:31:21.675: INFO/DEBUG(2880):  d30 0000000000000000  d31 0000000000000000
07-28 11:31:21.675: INFO/DEBUG(2880):  scr 80000012
07-28 11:31:21.753: INFO/DEBUG(2880):          #00  pc 005f0c38  /data/data/com.gnychis.coexisyst/lib/libtshark.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #01  pc 005c93f4  /data/data/com.gnychis.coexisyst/lib/libtshark.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #02  pc 00001dd2  /data/data/com.gnychis.coexisyst/lib/libwireshark_helper.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #03  pc 00001d9a  /data/data/com.gnychis.coexisyst/lib/libwireshark_helper.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #04  pc 00016e34  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #05  pc 000452c4  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #06  pc 0001bd98  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #07  pc 00022794  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #08  pc 00021634  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #09  pc 0005c5cc  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #10  pc 0005c7fc  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #11  pc 0005203e  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #12  pc 000520ca  /system/lib/libdvm.so
07-28 11:31:21.761: INFO/DEBUG(2880):          #13  pc 000525a0  /system/lib/libdvm.so
07-28 11:31:21.769: INFO/DEBUG(2880):          #14  pc 0004f9ec  /system/lib/libdvm.so
07-28 11:31:21.769: INFO/DEBUG(2880):          #15  pc 00010ed4  /system/lib/libc.so
07-28 11:31:21.769: INFO/DEBUG(2880):          #16  pc 000109c0  /system/lib/libc.so
07-28 11:31:21.769: INFO/DEBUG(2880): code around pc:
07-28 11:31:21.769: INFO/DEBUG(2880): 845f8c18 ebfda8d5 eaffffdf e5950014 e3500000
07-28 11:31:21.769: INFO/DEBUG(2880): 845f8c28 0affffdc ebfdc232 eaffffda e92d4070
07-28 11:31:21.769: INFO/DEBUG(2880): 845f8c38 e5904010 e1a05000 e3540000 0a000004
07-28 11:31:21.769: INFO/DEBUG(2880): 845f8c48 e5940000 ebfdac25 e5944004 e3540000
07-28 11:31:21.769: INFO/DEBUG(2880): 845f8c58 1afffffa e1a00005 e8bd4070 eafdd2ef
07-28 11:31:21.769: INFO/DEBUG(2880): code around lr:
07-28 11:31:21.769: INFO/DEBUG(2880): 845d13d8 e1a01003 eafe72bb e92d4010 e1a04000
07-28 11:31:21.769: INFO/DEBUG(2880): 845d13e8 e2800008 ebfe4c98 e5940000 ebfe4a3b
07-28 11:31:21.769: INFO/DEBUG(2880): 845d13f8 e5940004 e3500000 0a000001 e8bd4010
07-28 11:31:21.769: INFO/DEBUG(2880): 845d1408 eafe7322 e8bd8010 e92d4010 e1a04000
07-28 11:31:21.769: INFO/DEBUG(2880): 845d1418 ebfe690a e1a00004 e8bd4010 eafe46d2
07-28 11:31:21.769: INFO/DEBUG(2880): stack:
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa48  afd43724  /system/lib/libc.so
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa4c  00000004
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa50  00d417e0  [heap]
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa54  00000008
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa58  00000000
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa5c  afd10280  /system/lib/libc.so
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa60  00d417e0  [heap]
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa64  00000000
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa68  0024a330  [heap]
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa64  00000000
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa68  0024a330  [heap]
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa6c  00000001
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa70  00000008
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa74  00000000
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa78  00d3e778  [heap]
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa7c  8063b4af  /system/lib/libglib-2.0.so
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa80  df002777
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa84  e3a070ad
07-28 11:31:21.769: INFO/DEBUG(2880): #00 100ffa88  00d3e770  [heap]
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa8c  00000000
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa90  4184ff98
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa94  845d13f8  /data/data/com.gnychis.coexisyst/lib/libtshark.so
07-28 11:31:21.769: INFO/DEBUG(2880): #01 100ffa98  100ffaf0
07-28 11:31:21.769: INFO/DEBUG(2880):     100ffa9c  80d01dd5  /data/data/com.gnychis.coexisyst/lib/libwireshark_helper.so

First off, I'm not entirely sure how to properly interpret it, and if any of the memory addresses are actually useful for me. For example, can I take a memory address and determine which function the fault happened in? Based on the stack, which library was the one where the fault occurred?

The fault takes some time to happen, so stepping through code seems a little tricky. Is there a way to get a core dump or anything? I have attempted using log messages from the native code, but the log messages are not helpful enough for me to narrow the problem.

Answer 1

If your application is truly crashing in native code, I highly recommend the following:

1. Set a java breakpoint that will be hit after your native library has been loaded but before it starts executing.
2. Go to the terminal and launch the ndk's version of gdb (it's called "ndk-gdb"). Initially, just type "c" to continue.
3. Back in java, continue execution.
4. Once your app crashes, gdb will pause and you can examine the integrity of the stack in native code via the "bt" command (backtrace).

At any rate, you should be able to fully debug native code and catch problems via ndk-gdb. You can find specific documentation on the ndk's version of gdb in your local installation of the ndk (for example, on my machine it's here):

android-ndk-r6/documentation.html

Finally, here's a basic tutorial on gdb:

gdb tutorial

Hope this helps!