GitLab runner can not resolve host when running pipeline

Question

I have a dockerized GitLab and GitLab Runner installation, with following docker-compose.yml:

version: "3"

services:
  gitlab:
    image: gitlab/gitlab-ee:latest
    container_name: gitlab
    restart: always
    hostname: gitlab
    ports:
      - "45022:22"
      - "45080:80"
      - "45443:443"
    volumes:
      - /srv/gitlab/config:/etc/gitlab
      - /srv/gitlab/logs:/var/log/gitlab
      - /srv/gitlab/data:/var/opt/gitlab

  python-runner:
    image: gitlab/gitlab-runner:latest
    container_name: python-runner
    hostname: python-runner
    volumes:
      - /srv/python-runner/config:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - CI_SERVER_URL=http://gitlab/ci
      - RUNNER_TOKEN=myTokenCode
      - RUNNER_DESCRIPTION=Python 2.7.14
      - RUNNER_EXECUTOR=docker
      - DOCKER_IMAGE=python:2.7.14
    restart: always

I have registered the runner:

docker exec -it python-runner gitlab-runner register \
    --non-interactive \
    --url "http://gitlab/" \
    --registration-token "${GITLAB_REGISTRATION_TOKEN}" \
    --description "Python 2.7.14" \
    --executor "docker" \
    --docker-image python:2.7.14

The runner is listed in the Runners list:

I can ping the gitlab host from the python-runner:

» docker exec -it python-runner bash
root@python-runner:/# ping gitlab
PING gitlab (172.20.0.2) 56(84) bytes of data.
64 bytes from gitlab.gitlab_default (172.20.0.2): icmp_seq=1 ttl=64 time=0.112 ms
64 bytes from gitlab.gitlab_default (172.20.0.2): icmp_seq=2 ttl=64 time=0.055 ms
^C
--- gitlab ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.055/0.083/0.112/0.029 ms

But when running the pipeline, it fails:

`fatal: unable to access 'http://gitlab-ci-token:xxxxxxxxxxxxxxxxxxxx@gitlab/group/project.git/': Could not resolve host: gitlab

How is this possible? How can this be solved?

EDIT

It seems the architecture that I am creating is the following:

• gitlab running in gitlab docker container
• gitlab runner running in python-runner docker container
• docker-compose creates a private network gitlab_default (gitlab is the name of the project), and both the gitlab and python-runner containers can reach it other, by IP and by name.
• the python-runner uses the docker executor to spawn containers during CI (in the host?), based on python:2.7.14 as defined. I do not know how gitlab names these containers, let's call it ci-job
• the project is cloned in this ci-job container, by issuing a git clone. This fails because the ci-job container can not reach the gitlab container, since it is probably in a different network (default network?)

I have tried to force the python-runner to spawn containers in the same gitlab_default network, by using --docker-network-mode gitlab_default flag as follows:

docker exec -it python-runner gitlab-runner register \
    --non-interactive \
    --tag-list python-2.7.14 \
    --url "http://gitlab" \
    --registration-token "$(GITLAB_REGISTRATION_TOKEN)" \
    --name "Python 2.7.14" \
    --executor "docker" \
    --docker-image python:2.7.14 \
    --docker-network-mode gitlab_default

But it still does not work. I am not sure if that's the right flag, since it is poorly documented.

Two questions:

1. How can I see the containers that the executor is creating when running CI? Can I enter them and do some debugging there?
2. What is the relevant parameter to ensure that the containers spawned by the docker executor are in the same network as the gitlab container?

EDIT2

After some idle time, my jobs started working. It seems configuring --docker-network-mode did indeed work as expected.

Answer 1

I had the same problem with gitlab-runner not able to resolve host names. But we use Gitlab running on Debian server (not docker installation) and runners in Google Cloud connected with VPN.

What worked for me was to add dns addresses to the runners config.toml, like this:

  [runners.docker]
    dns = ["dns-1-ip", "dns-2-ip"]