How to get access to filesystem with "kubectl debug" (ephemeral containers)?

Question

If I do

POD=$($KUBECTL get pod -lsvc=app,env=production -o jsonpath="{.items[0].metadata.name}")
kubectl debug -it --image=mpen/tinker "$POD" -- zsh -i

I can get into a shell running inside my pod, but I want access to the filesystem for a container I've called "php". I think this should be at /proc/1/root/app but that directory doesn't exist. For reference, my Dockerfile has:

WORKDIR /app
COPY . .

So all the files should be in the root /app directory.

If I add --target=php then I get permission denied:

❯ cd /proc/1/root
cd: permission denied: /proc/1/root

How do I get access to the files?

Answer 1

The debug container has always its own filesystem, but - if the container runtime supports it - you should be able to use --target and access it via /proc/$PID/root just fine. One just needs to act as the exact same user (both $UID and $GID must match).

Example

First, prepare a container to be debugged:

kubectl run tested-pod --rm -it --image=busybox:latest --restart=Never \
  --overrides='
    {"spec": {
      "containers": [
        {
          "name":"tested-pod",
          "image":"busybox:latest",
          "stdin":true,
          "tty":true,
          "securityContext": {
            "runAsUser":1234,
            "runAsGroup":4321
          }
        }
      ]
    }
  }' \
  -- sh

This creates a new pod tested-pod where the container runs with UID=1234 and GID=4321, and gives you an interactive shell. Write something to the container's filesystem, so you can see it later:

touch /tmp/tested-pod-file
ls /tmp
# tested-pod-file

Next, open a new terminal and use the ephemeral container debug:

kubectl debug tested-pod -it --image=busybox:latest --target=tested-pod -- sh

At this moment, the debugging user is root, which can't access the FS, but we can already see the container's process:

id
# uid=0(root) gid=0(root) groups=0(root),10(wheel)
ls /proc/1/root
# ls: /proc/1/root: Permission denied
ps -o 'pid,user,group,comm'
# PID   USER     GROUP    COMMAND
#     1 1234     4321     sh
#     9 root     root     sh
#    25 root     root     ps

Now you gotta create a local group and user (name doesn't matter) and use it:

addgroup -g 4321 karel
adduser -HD -u 1234 -G karel karel
su karel

In the resulting shell you should be able to access the filesystem, even for writing (at least it works in my case):

ls /proc/1/root/tmp/
# tested-pod-file
touch /proc/1/root/tmp/debugger-file
ls /proc/1/root/tmp/
# debugger-file    tested-pod-file

You can verify this worked from the original container's shell:

ls /tmp
debugger-file    tested-pod-file