Clone a secure git repo in Kubernetes pod

Question

I've run into an interesting situation where I need to clone a private github repo into a docker container that I'm running in Kubernetes. Originally I tried using a gitRepo mount, however, having an OAuth key in my deployment manifest is unacceptable and I would like to use a repo deploy key rather an OAuth key attached to my GitHub account.

Ideally I would use a gitRepo mount that auths using a secret, but this feature isn't available at the time of writing.

Constraints

I need the following:

• A repo inside the container that I can pull intermittently while my container's running
• The repo must be accessed using a GitHub deploy key
• The key must be kept secure (in a Kubernetes secret) and not stored in the docker image
• The repo must be shared between two containers in the same pod—one that writes and one that reads

Possible Solutions:

Mount SSH keys as secrets and clone:

I tried cloning the repo into an emptydir with a bash script running in a separate pod (this script has to run anyway, I'm using it for other things as well), however I then ran into the issue of getting ssh keys into the pod. This question is about this issue, however it doesn't seem to have a way to do it. I was able to get the keys in with a secret mount, but then the permissions are set to 777. To get around this, I mounted the keys into a /test/ directory and then tried to cp them into /root/.ssh/. This gave me these strange errors:

cp: '/test/id_rsa' and '/root/.ssh/id_rsa' are the same file
cp: '/test/id_rsa.pub' and '/root/.ssh/id_rsa.pub' are the same file

I also tried using cat and piping them to their files but that didn't work. At first it gave me these errors when I had the paths wrong:

cat: /keys/id_rsa: input file is output file
cat: /keys/id_rsa.pub: input file is output file

Once I fixed the paths, it did nothing and silently failed. kubectl execing into the containter showed no files in /root/.ssh/.

I think I've pretty much reached the bottom of this path so I don't think it will be the solution.

Configuring ssh to ignore key permissions

If SSH has a way to ignore the permissions on keys—by default it enforces 644 or less I believe—then the above solution could be possible. I'm surprised I haven't found any ways to do this, but my google-fu always turned up results saying that you just have to set the permissions properly.

Some other way of getting the keys into the container securely

Ideally, I would like to have a key in the container for potential future expansions of this project using other repos. There could be some other way of doing this that I haven't thought of or tried.

Clone using OAuth key within the container

I've thought about trying to use an OAuth key in an environment variable and then using that to clone the repo by HTTPS. This is less than ideal but if it works I'll take it. The only thing preventing me from doing this now is that I can't use a deploy key. If there is a way to use OAuth with a deploy key, I haven't found it yet but it could be the solution if someone knows more.

Cloning in the docker image

There isn't anything in the repo that I would be uncomfortable having in the docker image so I could go this route. The problem with this is that I need to be able to pull down updates to the repo. If I put it in the container, I won't be able to pull without my keys. There could be some workaround to this that I haven't tried.

I'm at the point where I feel like I have nothing left to try so any suggestions could be worth a shot.

---

Similar Questions

This question is very similar to what I'm trying to do, however I feel that since they aren't using Kubernetes and I am, it's worth posting separately because of secrets and the different methods of mounting files.

This question talks about getting SSH keys into containers in Kubernetes, however I'm not dead-set on using git over SSH so I think this should be it's own question.

Answer 1

Try to mount your secret containing your deploy key like this:

volumeMounts:
  - mountPath: /root/.ssh
    name: ssh-key
volumes:
- name: ssh-key
  secret:
    secretName: ssh-key
    defaultMode: 256

Here is the full example of how I'm using it:

apiVersion: batch/v2alpha1
kind: ScheduledJob
metadata:
  name: transporter
spec:
  schedule: 0 5 * * *
  jobTemplate:
    spec:
      template:
        spec:
          nodeSelector:
            role: mysqldump
          containers:
          - name: transporter
            image: camil/mysqldump
            command: ["/bin/bash", "-c"]
            args:
              - ssh-keyscan -t rsa $TARGET_HOST > ~/.ssh/known_hosts && ssh -i /root/.ssh/private/id_rsa $LINUX_USER@$TARGET_HOST 'mkdir mysqldump || true' && scp -i /root/.ssh/private/id_rsa /mysqldump/* $LINUX_USER@$TARGET_HOST:/home/$LINUX_USER/mysqldump
            env:
              - name: TARGET_HOST
                valueFrom:
                  configMapKeyRef:
                    name: transporter
                    key: target.host
              - name: LINUX_USER
                valueFrom:
                  configMapKeyRef:
                    name: transporter
                    key: linux.user
            imagePullPolicy: Always
            volumeMounts:
              - mountPath: /mysqldump
                name: mysqldump
              - mountPath: /root/.ssh/private
                name: ssh-key
          volumes:
            - name: mysqldump
              hostPath:
                path: /home/core/mysqldump
            - name: ssh-key
              secret:
                secretName: ssh-key
                defaultMode: 256
          restartPolicy: OnFailure