gdb does not hit any breakpoints when I run it from inside Docker container

Question

The problem

I am able to set and reach a breakpoint if I compile and run from the host, but if I do it from within the docker container gdb does not hit the breakpoints that were set.

Steps to reproduce (all snippets are copy-paste ready)

Create a docker file:

cat << EOF > Dockerfile
FROM ubuntu
RUN apt-get update
RUN apt-get install -y build-essential gdb
EOF

Build an image and run an interactive session in it:

docker build -t gdb_problem_testing . && docker run --rm -it  gdb_problem_testing bash

From inside of the container create small main.cpp, compile, and run gdb:

cat <<EOF > main.cpp && g++ -g main.cpp && gdb -ex 'break 5' -ex 'run' ./a.out
#include <iostream>

int main(int argc, const char *argv[])
{
    std::cout << "hi\n";
    return 0;
}
EOF

Observe the gdb output:

GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
[Skipped gdb greeting]
Reading symbols from ./a.out...done.
Breakpoint 1 at 0x40078c: file main.cpp, line 5.
1   #include <iostream>
2   
3   int main(int argc, const char *argv[])
4   {
5       std::cout << "hi\n";
6       return 0;
7   }
Starting program: /a.out 
hi
During startup program exited normally.
(gdb) 

From the output one can see that the breakpoint was not hit, although the program was executed (printed 'hi') and exited successfully. I guess the most important thing here is that the program did run and that the message During startup program exited normally is an anomalous behavior (according to GDB ignores my breakpoints )

Question

What is preventing gdb from setting the breakpoint and how to fix this?

What I tried so far

1. As suggested here, I tried to change a line in /etc/apparmor.d/docker (I did it in the host): substitute profile docker-default flags=(attach_disconnected,mediate_deleted) { by profile docker-default flags=(attach_disconnected,mediate_deleted,complain) {. Then run the docker container, compile, and gdb. The result was the same: During startup program exited normally.

2. As suggested in another answer, from within the container, I tried to do strace -f -o syscall.txt gdb ./a.out, but I get the following error:

   strace: test_ptrace_setoptions_followfork: PTRACE_TRACEME doesn't work: Permission denied
   strace: test_ptrace_setoptions_followfork: unexpected exit status 1
   

   but I do not understand how to work this around. I tried to start the container as root: sudo docker run --rm -it gdb_problem_testing bash and then tried the strace -- this gave me the same error. I must admit I do not understand how the user privileges are managed by the docker, i.e. what user rights the root inside of the container has and from whom does it inherit the rights (from the docker daemon?). Since I able to hit the breakpoint, when I run gdb in the host, I suspect that my problem would boil down to the user rights, but I do not know how to approach it.

3. In the host I tried to do echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope as suggested in yet another answer.

Answer 1

update 2020.01.04: Use the answer given by Kevin W Matthews --- it's better because it grants the necessary individual capabilities without elevating the entire container.

---

tldr; use

docker run --privileged

Longer: I was having some problems with gdb in docker---it was attempting (and failing) to disable address space layout randomization---but only on docker-machine, not on my native linux host.

When gdb failed to disable ASLR, all of my breakpoints would be ignored. Using the --privileged flag fixed my issue. Your mileage may vary.

Answer 2

Rather than elevating the entire container, I was able to use the option

--security-opt seccomp=unconfined

to fix address space randomization problems.

Some also recommend enabling the ptrace capability with

--cap-add=SYS_PTRACE

but this didn't seem to have any effect for me.

Here are the same settings for Docker compose:

security_opt:
  - seccomp:unconfined
cap_add:
  - SYS_PTRACE

Details are taken from this Stack Overflow post.

Answer 3

thanks the answer from @rubicks.

and if you can't use --privileged option(e.g. you are using a container from cloud), you can print stacktrace from your program:

How to automatically generate a stacktrace when my gcc C++ program crashes