gitlab in docker behind traefik proxy fails (usually)

Question

I have several web sites running in docker with LetsEncrypt credentials and routed via traefik. I would like to run a local gitlab-ce in docker similarly with LetsEncrypt and traefik.

So I added this to my traefik.toml file:

[[acme.domains]]
  main = "gitlab.mydomain.com"

And this to config/gitlab.rb:

external_url "http://gitlab.mydomain.com"

And I start gitlab with:

docker run -d --restart=always \
     --hostname gitlab.mydomain.com \
     --expose 80 \
     --volume /srv/gitlab/config:/etc/gitlab \
     --volume /srv/gitlab/data:/var/opt/gitlab \
     --volume /var/log/gitlab:/var/log/gitlab \
     --label traefik.frontend.rule=Host:gitlab.mydomain.com \
     --name gitlab gitlab/gitlab-ce:latest

Going to https://gitlab.mydomain.com/ I get a secure site with a LetsEncrypt generated certificate, but the site doesn't load:

Internal Server Error

When I reload the page I see this in docker logs gitlab -f:

==> /var/log/gitlab/sshd/current <==
2017-02-12_16:51:31.00446 Bad protocol version identification 'GET / HTTP/1.1' from 172.17.0.8 port 41138
2017-02-12_16:51:31.26238 Bad protocol version identification 'GET /favicon.ico HTTP/1.1' from 172.17.0.8 port 41140

Searching for /error/i in the logs I see several things that could be issues (lots of errors reported in zruby/gems/2.3.0/gems/redis-3.2.2z) but no "smoking gun" AFAICT.

And to top off the craziness, about every ten or so (random) times that I run docker restart gitlab the site comes up perfectly. I've been tempted to just leave it up, but therein lies madness...

How can I get it to come up reliably? Or how can I debug this more completely?

Answer 1

This answer probably comes way too late for you, but I ran into the same issue and was able to solve it.

The important clue is that the log errors are by the sshd daemon!

Traefik will, by default, pick the first port exposed by the container (by the Dockerfile, not the ports you manually expose!). In case of the Gitlab container, this is the ssh port 22.

So Traefik will direct the web requests to Gitlab's SSH daemon.

To fix this, you need to set the port for Traefik explicitly, with a label:

Traefik 1.x:

labels:
    ...
    - traefik.port=80

Traefik 2.x:

labels:
    - traefik.http.services.<your-service-name>.loadbalancer.server.port=80

Answer 2

i've used sameersbn's docker-compose and added the following docker-compose.override.yml in the same directory.

version: "2"

services:
    gitlab:
      labels:
        - "traefik.frontend.rule=Host:git.schulz.codes"
        - "traefik.port=80"
        - "traefik.enable=true"
        - "traefik.frontend.entryPoints=http,https"

this keeps working quiet nicely with the following traefik docker-compose

version: "2"

services:
  proxy:
    restart: always
    image: traefik
    container_name: traefik
    command: --web --docker --docker.domain=docker.localhost --logLevel=DEBUG
    ports:
      - "8080:8080"
      - "80:80"
      - "443:443"
    volumes:
      - ./traefik.toml:/etc/traefik/traefik.toml
      - /var/run/docker.sock:/var/run/docker.sock
      - ./data:/etc/traefik/acme:rw

and this traefik.toml

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
    entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
defaultEntryPoints = ["http", "https"]
[acme]
email = "[email protected]"
storageFile = "/etc/traefik/acme/acme.json"
entryPoint = "https"
OnHostRule = true
[[acme.domains]]
  main = "domain.com"
  sans = ["gitlab.domain.com"]
[web]
address = ":8080"
[docker]
endpoint = "unix:///var/run/docker.sock"
domain = "docker.localhost"
watch = true
exposedbydefault = true