How to runtime debug shared libraries?

Question

You just need to call gdb with the executable (it does not matter if it is yours or a 3rd party one). Here is an example where I debug the ls command and set a breakpoint in the (shared) c library. This example uses gdb 6.8 which supports deferred (pending) breakpoints which makes this easy:

gdb /bin/ls
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
(no debugging symbols found)
(gdb) b write
Function "write" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (write) pending.
(gdb) r
Starting program: /bin/ls
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
(no debugging symbols found)
[Thread debugging using libthread_db enabled]
(no debugging symbols found)
(no debugging symbols found)
[New Thread 0x7f98d2d23780 (LWP 7029)]
[Switching to Thread 0x7f98d2d23780 (LWP 7029)]

Breakpoint 1, 0x00007f98d2264bb0 in write () from /lib/libc.so.6
(gdb)

As you can see gdb automatically manages all threads used by the executable. You don't have to do anything special for threads there. The breakpoint will work in any thread.

Alternatively if you want to attach the debugger to an already running application (I use tail -f /tmp/ttt here as an example):

ps ux | grep tail
lothar    8496  0.0  0.0   9352   804 pts/3    S+   12:38   0:00 tail -f /tmp/ttt
lothar    8510  0.0  0.0   5164   840 pts/4    S+   12:39   0:00 grep tail

gdb
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
(no debugging symbols found)
(gdb) attach 8496
Attaching to program: /usr/bin/tail, process 8496
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x7f24853f56e0 (LWP 8496)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/ld-linux-x86-64.so.2...
(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
(no debugging symbols found)
0x00007f2484d2bb50 in nanosleep () from /lib/libc.so.6
(gdb) b write
Breakpoint 1 at 0x7f2484d57bb0
(gdb) c
Continuing.
[Switching to Thread 0x7f24853f56e0 (LWP 8496)]

Breakpoint 1, 0x00007f2484d57bb0 in write () from /lib/libc.so.6
(gdb)

Normally the procedure for debugging a shared library is much the same as for debugging an executable - the main difference is that you may be unable to set a breakpoint until the shared library is loaded into memory. You attach the debugger to the main executable.

If you are debugging an application that is not owned by you, but is using your module in a plugin architecture, you still use the same method. Make sure (as always) that you have debugging information available for your shared library. In windows, you would generate a .pdb file. With gcc, I think you specify a special compiler flag (-g?) to ensure that debugging information is supplied. You attach the debugger to the third party application.

Another example further to lothar's answer:

I'm running tests on a dynamic library test.so (compiled from test.c) in Linux using python and python's unit-testing library unittest called tests/test_pwmbasic.py. (naming scheme is a bit monotone, I realise that now)

~/my/test/path/
    tests/
        __init__.py
        test_pwmbasic.py
    test.c
    test.so

I want to debug what's in test.so from stimulus in test_pwmbasic.py. So this is how I got it working...

$ cd ~/my/test/path
$ gdb $(which python)
   ... gdb blah ...
(gdb) b test.c:179
(gdb) run
>>> from tests.test_pwmbasic import *
>>> import unittest
>>> unittest.main()
   ... unittest blah ...
Breakpoint 1, pwmTest_setDutyCycles (dutyCycles=0x7ffff7ece910) at ./test.c:179
(gdb) print pwm_errorCode
$1 = PWM_ERROR_NONE

and now I want to marry gdb

note: test.c also includes ../pwm.c, so I can also breakpoint within that library with

(gdb) b pwm.c:123

I remember testing shared libraries by creating a mock app that used it. If you are willing to do a lot of work, you could create a second, mock shared library that just collects information about how the library is being used by the third party app, and then have your mock app replay that information.

Of course, never doubt the power of well placed printf and fprintf calls.

You could try compiling and linking the library statically to debug it.
If your bug only shows up when it's compiled as shared, than that might give you some clues.