How to set breakpoint in gdb when attach to another process

Question

I have a C program which a really complicated script is written to run it. I need to debug this program using gdb. I have tried to run the script and attach gdb to it's process, but then I'm not able to set breakpoints that I need:

$ gdb median.o 27944
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from median.o...done.
Attaching to program: median.o, process 27944
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/ld-2.19.so...done.
done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x00007fc376e9cbfa in ?? ()
(gdb) break median.c:10
Cannot access memory at address 0x40059d

I also tried this:

$gdb -p 28303
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 28303
Reading symbols from /bin/dash...(no debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.19.so...done.
done.
Loaded symbols for /lib/x86_64-linux-gnu/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from /usr/lib/debug//lib/x86_64-linux-gnu/ld-2.19.so...done.
done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x00007fe918e50bfa in wait4 () at ../sysdeps/unix/syscall-template.S:81
81  ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) break median.c:10
No source file named median.c.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (median.c:10) pending.
(gdb) continue
Continuing.
Please enter 3 numbers separated by spaces > 6 is the median
[Inferior 1 (process 28303) exited normally]

So it continues without stopping at the breakpoint. It is worth mentioning that if I call run median.o < input from gdb itself, it works.

How can I set breakpoints on a C program which is being run by a script?

Answer 1

This is a classic problem that people have with gdb. It's so common that you'd think it would have a handy name!

There are a few solutions to the problem, some time-tested and some relatively more experimental.

• If the program to debug (in gdb lingo, "the inferior") is long-running -- for example, a GUI or a server of some kind -- then the simplest way is to just run the script, wait for the inferior to start, and then attach to it. You can attach using the PID, either with gdb -p PID or using attach PID at the gdb prompt.

• If the program is short-lived, then another classic approach is to add a call to sleep early in the program's startup; say as the first line of main. Then, continue with the attach plan.

Those are the classic ways. But now let's talk about the more fun stuff.

gdb has a multi-inferior mode, where it can debug multiple processes at once. This mode, IME, is still a bit fragile, but I've had some success with it.

First you put gdb into the correct mode:

set detach-on-fork off
set non-stop off
set pagination off

(If you have an older gdb you will also need set target-async on).

Now you can debug the shell, something like:

$ gdb --args /bin/sh /path/to/my/script
(gdb) [... set the mode as above ...]
(gdb) break some_function_in_my_inferior

Now run should start the script, and automatically attach gdb to each child process that's created; eventually stopping at the breakpoint.

There's still one more way. A long time ago, there was a kernel patch to add "global breakpoints", plus a gdb patch to work with this feature. As far as I know, none of this was ever merged. But, I wrote a variant in my gdb helpers project.

There is a new command there called preattach. What it does is use SystemTap to watch for an exec of a specified program; then it pauses this program while gdb attaches to it.