How do I enable cloning over SSH for a Gitlab runner?

Question

I am having some trouble cloning large repositories over HTTP on my Windows Gitlab runner. I've tried several methods to do shallow clones or disable clone compression. Still no luck.

Cloning the same repository over SSH works great as a temporary solution and I would like to get this working on our Gitlab CI process.

The issue now stands where I have no idea how to use SSH as a clone method for the gitlab-multi-runner. It just seems to use HTTP as a default, and my only options regarding cloning is whether it will do a full clone or a fetch.

Can someone explain how I could get that clone/fetch to work on a runner over SSH instead of HTTP?

Gitlab Version: GitLab Community Edition 8.10.7

Answer 1

As a newcomer to gitlab, I've managed to hack a workaround to this issue as I also haven't found a built-in way to change the default cloning process (although here is a recent comment about how it can be done).

By disabling the automatic cloning process, you can effectively override its behavior completely by simply writing your own cloning process in a before_script. Only for the purposes of example does the below show how to accomplish this for HTTP cloning but could be adapted for ssh cloning (if you're trying to use HTTP cloning you should use the built-in cloning process and the config.toml):

1. Create a new user called "gitlab-runner" and generate their user auth token for later use (or in your case, you would generate ssh keys).

2. Disable cloning process for runner by adding the following variable in either your project or group settings: .../settings/ci_cd

   key: GIT_STRATEGY

   value: none

3. Clone your repo in a before_script such as:

 before_script:   ## clean the working directory   - BUILD_DIR=/home/gitlab-runner/builds/$RUNNER_TOKEN/0   - CLONE_DIR="$BUILD_DIR/$CI_PROJECT_PATH"   - cd $BUILD_DIR   - rm -rf $CLONE_DIR   - mkdir -p $CLONE_DIR    ## clone the project each time (inefficient, consider performing fetch instead if it already exists)   - git clone http://gitlab-runner:$GITLABRUNNER_USER_AUTH_TOKEN@server:8888/${CI_PROJECT_PATH}.git $CLONE_DIR   - cd $CLONE_DIR 

Note: Here are the relevant variables I also configured in step 2 rather than hard coding them in the script:

• RUNNER_TOKEN: "Runner Token" value listed in the Admin "Runners" menu for the particular runner you are trying to run.
• GITLABRUNNER_USER_AUTH_TOKEN: This is the auth token you generated in step 1.

Further Reading:

You can avoid the fake account approach taken above by instead issuing Deploy Keys. Or if security implications of access to any project is a concern, Deploy Tokens are an alternative with more security control. For comparison, see the docs:

Deploy keys are shareable between projects that are not related or don’t even belong to the same group. Deploy tokens belong to either a project or a group.

A deploy key is an SSH key you need to generate yourself on your machine. A deploy token is generated by your GitLab instance, and is provided to users only once (at creation time).

A deploy key is valid as long as it’s registered and enabled. Deploy tokens can be time-sensitive, as you can control their validity by setting an expiration date to them.

You can’t log in to a registry with deploy keys, or perform read / write operations on it, but this is possible with deploy tokens. You need an SSH key pair to use deploy keys, but not deploy tokens.

Answer 2

According to:

https://docs.gitlab.com/ee/ci/ssh_keys/README.html

You need to:

1. Create a new SSH key pair with ssh-keygen
2. Add the private key as a Secret Variable to the project
3. Run the ssh-agent during job to load the private key.

Example gitlab_ci.yml:

before_script:   # Install ssh-agent if not already installed, it is required by Docker.   # (change apt-get to yum if you use a CentOS-based image)   - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'    # Run ssh-agent (inside the build environment)   - eval $(ssh-agent -s)    # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store   - ssh-add <(echo "$SSH_PRIVATE_KEY")    # For Docker builds disable host key checking. Be aware that by adding that   # you are suspectible to man-in-the-middle attacks.   # WARNING: Use this only with the Docker executor, if you use it with shell   # you will overwrite your user's SSH config.   - mkdir -p ~/.ssh   - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'   # In order to properly check the server's host key, assuming you created the   # SSH_SERVER_HOSTKEYS variable previously, uncomment the following two lines   # instead.   # - mkdir -p ~/.ssh   # - '[[ -f /.dockerenv ]] && echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts'