How can gitlab-CI install private python packages from a gitlab dependency that also refers to gitlab repositories

Question

This question is about how to manage access for private gitlab python packages with nested dependencies on other private gitlab packages. This assumes all the access is via direct git repository patterns and not a private package repository.

package-a is in a private gitlab repository and it depends on package-b, which depends on package-c and they are also in a private gitlab repository.

package-a has a pyproject.toml like this:

[tool.poetry]
name = "package-a"
repository = "https://gitlab.com/org/package_a.git"

[tool.poetry.dependencies]
python = "^3.6"
package-b = {git = "ssh://[email protected]/org/package_b.git", tag = "0.1.0"}

package-b has a pyproject.toml like this:

[tool.poetry]
name = "package-b"
repository = "https://gitlab.com/org/package_b.git"

[tool.poetry.dependencies]
python = "^3.6"
package-c = {git = "ssh://[email protected]/org/package_c.git", tag = "0.1.0"}

Any user with the right org membership on gitlab and an ssh-key can use poetry to install package-a and it's dependency on package-b and then it's dependency on package-c, all into a python venv on a development laptop. The ssh protocol access also works for docker builds (with experimental features for ssh mounts).

However, the same projects with private dependencies are not installed in gitlab-CI runners, because they lack ssh access. (Is there any secure way to enable that?)

Assuming the gitlab-CI runners must use an access token to clone the private gitlab repositories, a sed script is applied to the pyproject.toml file for project-a, so the gitlab-CI runner can clone package-b to discover that it depends on package-c; the sed script changes the ssh to https access for project-b by editing the project-a dependency spec in pyproject.toml, i.e.

sed -i -e 's#ssh://[email protected]/org#https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/org#g' pyproject.toml

The CI_JOB_TOKEN is an environment variable provided by the gitlab-CI runner. It is securely managed by the gitlab-CI runner. So, gitlab-CI runner can now clone the project-b repository somewhere. The same sed trick might work if it could be applied to this project-b repository somewhere, but this is now in the hands of poetry and it cannot be touched. So project-b has a git+ssh dependency on project-c and the gitlab-CI runner fails to install project-c because it has no git+ssh credentials to clone it.

Thus, the private package dependency chain works for development and docker build on a laptop with git+ssh access, but it's all broken on gitlab-CI. What's the better practice to manage this private package access across all these build environments?

Answer 1

These snippets are based on:

• https://docs.gitlab.com/ee/ci/ssh_keys/
• https://gitlab.com/gitlab-examples/ssh-private-key
• https://gitlab.com/gitlab-examples/ssh-private-key/issues/7
• https://gitlab.com/gitlab-examples/ssh-private-key/issues/1

ssh-keygen -o -t rsa -b 4096 -C "[email protected]"

# output to something like ~/.ssh/gitlab_ci_rsa
# do not add any passphrase

# once created, copy the private key to the clipboard, e.g.
cat ~/.ssh/gitlab_ci_rsa | base64 -w0 > tmp.txt
xclip -sel clip < tmp.txt 

The public key is used as a private deploy key, which is enabled from a project settings page, e.g.

• https://gitlab.com/org/project-a/settings/repository
• https://gitlab.com/org/project-b/settings/repository
• https://gitlab.com/org/project-c/settings/repository

The private key is pasted into the gitlab-CI variable SSH_PRIVATE_KEY and gitlab should be able to mask it (when it is base64 encoded). Then the .gitlab-ci.yml file can add this private key to the ssh-agent using:

before_script:
  - apt-get update -y -qq && apt-get install -y -qq git make openssh-client
  - eval $(ssh-agent -s)
  ## Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent store
  - ssh-add <(echo "$SSH_PRIVATE_KEY" | base64 --decode)
  ## Create the SSH directory and give it the right permissions
  - mkdir -p ~/.ssh
  - chmod 700 ~/.ssh
  - ssh-keyscan gitlab.com >> ~/.ssh/known_hosts
  - chmod 644 ~/.ssh/known_hosts

The gitlab documentation does not use base64 encoding, but it's essential to copying all of the private key into the variable and that avoids a prompt for a passphrase from ssh-add.