How to customise config.toml on Kubernetes?

Question

I'm have a Gitlab cloud connected to a k8s cluster running on Google (GKE). The cluster was created via Gitlab cloud.

I want to customise the config.toml because I want to fix the cache on k8s as suggested in this issue.

I found the config.toml configuration in the runner-gitlab-runner ConfigMap. I updated the ConfigMap to contain this config.toml setup:

  config.toml: |
    concurrent = 4
    check_interval = 3
    log_level = "info"
    listen_address = '[::]:9252'
    [[runners]]
      executor = "kubernetes"
      cache_dir = "/tmp/gitlab/cache"
      [runners.kubernetes]
        memory_limit = "1Gi"
        [runners.kubernetes.node_selector]
          gitlab = "true"
        [[runners.kubernetes.volumes.host_path]]
          name = "gitlab-cache"
          mount_path = "/tmp/gitlab/cache"
          host_path = "/home/core/data/gitlab-runner/data"

To apply the changes I deleted the runner-gitlab-runner-xxxx-xxx pod so a new one gets created with the updated config.toml.

However, when I look into the new pod, the /home/gitlab-runner/.gitlab-runner/config.toml now contains 2 [[runners]] sections:

listen_address = "[::]:9252"
concurrent = 4
check_interval = 3
log_level = "info"

[session_server]
  session_timeout = 1800

[[runners]]
  name = ""
  url = ""
  token = ""
  executor = "kubernetes"
  cache_dir = "/tmp/gitlab/cache"
  [runners.kubernetes]
    host = ""
    bearer_token_overwrite_allowed = false
    image = ""
    namespace = ""
    namespace_overwrite_allowed = ""
    privileged = false
    memory_limit = "1Gi"
    service_account_overwrite_allowed = ""
    pod_annotations_overwrite_allowed = ""
    [runners.kubernetes.node_selector]
      gitlab = "true"
    [runners.kubernetes.volumes]

      [[runners.kubernetes.volumes.host_path]]
        name = "gitlab-cache"
        mount_path = "/tmp/gitlab/cache"
        host_path = "/home/core/data/gitlab-runner/data"

[[runners]]
  name = "runner-gitlab-runner-xxx-xxx"
  url = "https://gitlab.com/"
  token = "<my-token>"
  executor = "kubernetes"
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
  [runners.kubernetes]
    host = ""
    bearer_token_overwrite_allowed = false
    image = "ubuntu:16.04"
    namespace = "gitlab-managed-apps"
    namespace_overwrite_allowed = ""
    privileged = true
    service_account_overwrite_allowed = ""
    pod_annotations_overwrite_allowed = ""
    [runners.kubernetes.volumes]

The file /scripts/config.toml is the configuration as I created it in the ConfigMap. So I suspect the /home/gitlab-runner/.gitlab-runner/config.toml is somehow updated when registering the Gitlab-Runner with the Gitlab cloud.

If if changing the config.toml via the ConfigMap does not work, how should I then change the configuration? I cannot find anything about this in Gitlab or Gitlab documentation.

Answer 1

Inside the mapping you can try to append the volume and the extra configuration parameters:

# Add docker volumes
cat >> /home/gitlab-runner/.gitlab-runner/config.toml << EOF

      [[runners.kubernetes.volumes.host_path]]
        name = "var-run-docker-sock"
        mount_path = "/var/run/docker.sock"
EOF

I did the runner deployment using a helm chart; I guess you did the same, in the following link you will find more information about the approach I mention: https://gitlab.com/gitlab-org/gitlab-runner/issues/2578

If after appending the config your pod is not able to start, check the logs, I did test the appending approach and had some errors like "Directory not Found," and it was because I was appending in the wrong path, but after fixing those issues, the runner works fine.