How can I properly setup basic traefik reverse proxy?

Question

Assume my current public IP is 101.15.14.71, I have a domain called example.com which I configured using cloudflare and I created multiple DNS entry pointing to my public ip.

Eg:

1) new1.example.com - 101.15.14.71
2) new2.example.com - 101.15.14.71
3) new3.example.com - 101.15.14.71

Now, Here's my example project structure,

├── myapp
│   ├── app
│   │   └── main.py
│   ├── docker-compose.yml
│   └── Dockerfile
├── myapp1
│   ├── app
│   │   └── main.py
│   ├── docker-compose.yml
│   └── Dockerfile
└── traefik
    ├── acme.json
    ├── docker-compose.yml
    ├── traefik_dynamic.toml
    └── traefik.toml

Here I have two fastAPIs (i.e., myapp, myapp1)

Here's the example code I have in main.py in both myapp and myapp1, Its exactly same but return staement is different that's all

from fastapi import FastAPI
app = FastAPI()
@app.get("/")
def read_main():
    return {"message": "Hello world for my project myapp"}

Here's my Dockerfile for myapp and myapp1, here too both are exactly same but the only difference is I deploy myapp on 7777 and myapp1 on 7778 in different containers

FROM ubuntu:latest

ARG DEBIAN_FRONTEND=noninteractive
RUN apt update && apt upgrade -y
RUN apt install -y -q build-essential python3-pip python3-dev

# python dependencies
RUN pip3 install -U pip setuptools wheel
RUN pip3 install gunicorn fastapi uvloop httptools "uvicorn[standard]"

# copy required files
RUN bash -c 'mkdir -p /app'
COPY ./app /app


ENTRYPOINT /usr/local/bin/gunicorn \
    -b 0.0.0.0:7777 \ # this line I use for myapp dockerfile
    -b 0.0.0.0:7778 \ # this line I change for myapp1 dockerfile
    -w 1 \
    -k uvicorn.workers.UvicornWorker app.main:app \
    --chdir /app

Here's my docker-compose.yml file for myapp and myapp1, here also I have exactly same but only difference is I change the port,

services:
  myapp:  # I use this line for myapp docker-compose file
  myapp1: # I use this line for myapp1 docker-compose file
    build: .
    restart: always
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=traefik_public"

      - "traefik.backend=myapp" # I use this line for myapp docker-compose file
      - "traefik.backend=myapp1" # I use this line for myapp1 docker-compose file


      - "traefik.frontend.rule=Host:new2.example.com" # I use this for myapp compose file
      - "traefik.frontend.rule=Host:new3.example.com" # I use this for myapp1 compose file

      - "traefik.port=7777" # I use this line for myapp docker-compose file
      - "traefik.port=7778" # I use this line for myapp1 docker-compose file
    networks:
      - traefik_public

networks:
  traefik_public:
    external: true

Now coming to traefik folder,

1. acme.json # I created it using nano acme.json command with nothing in it, but did chmod 600 acme.json for proper permissions.

2. traefik_dynamic.toml

[http]
  [http.routers]
    [http.routers.route0]
      entryPoints = ["web"]
      middlewares = ["my-basic-auth"]
      service = "api@internal"
      rule = "Host(`new1.example.com`)"
      [http.routers.route0.tls]
        certResolver = "myresolver"

[http.middlewares.test-auth.basicAuth]
  users = [
    ["admin:your_encrypted_password"]
  ]

3. traefik.toml

[entryPoints]
  [entryPoints.web]
    address = ":80"
    [entryPoints.web.http]
      [entryPoints.web.http.redirections]
        [entryPoints.web.http.redirections.entryPoint]
          to = "websecure"
          scheme = "https"

  [entryPoints.websecure]
    address = ":443"

[api]
  dashboard = true

[certificatesResolvers.myresolver.acme]
  email = "[email protected]"
  storage= "acme.json"
  [certificatesResolvers.myresolver.acme.httpChallenge]
    entryPoint = "web"

[providers]
  [providers.docker]
    watch = true
    network = "web"
  [providers.file]
    filename = "traefik_dynamic.toml"

4. docker-compose.yml

services:
  traefik:
    image: traefik:latest
    ports:
      - 80:80
      - 443:443
      - 8080:8080
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik.toml:/traefik.toml
      - ./acme.json:/acme.json
      - ./traefik_dynamic.toml:/traefik_dynamic.toml
    networks:
      - web

networks:
  web:

These are the details about my files, what I am trying to achieve here is,

I want to setup traefik and traefik dashboard with basic authentication, and I deploy two of my fastapi services,

• myapp 7777, I need to access this app via new2.example.com
• myapp1 7778, I need to access this app via new3.example.com
• traefik dashboard, I need to access this via new1.example.com

All of these should be https and also has certification autorenew enabled.

I got all these from online articles for latest version of traefik. But the problem is this is not working. I used docker-compose to build and deploy the traefik and I open the api dashboard. It is asking for password and user (basic auth I setup) I entered my user details I setup in traefik_dynamic.toml but its not working.

Where did I do wrong? Please help me correcting mistakes in my configuration. I am really interested to learn more about this.

Error Update:

traefik_1  | time="2021-06-16T01:51:16Z" level=error msg="Unable to obtain ACME certificate for domains \"new1.example.com\": unable to generate a certificate for the domains [new1.example.com]: error: one or more domains had a problem:\n[new1.example.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://new1.example.com/.well-known/acme-challenge/mu85LkYEjlvnbDI-wM2xMaRFO1QsPDNjepTDb47dWF0 [2606:4700:3032::6815:55c4]: 404\n" rule="Host(`new1.example.com`)" routerName=api@docker providerName=myresolver.acme

traefik_1  | time="2021-06-16T01:51:19Z" level=error msg="Unable to obtain ACME certificate for domains \"new2.example.com\": unable to generate a certificate for the domains [new2.example.com]: error: one or more domains had a problem:\n[new2.example.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://new2.example.com/.well-known/acme-challenge/ykiCAEpJeQ1qgVdeFtSRo3q-ATTwgKdRdGHUs2kgIsY [2606:4700:3031::ac43:d1e9]: 404\n" providerName=myresolver.acme routerName=myapp1@docker rule="Host(`new2.example.com`)"

traefik_1  | time="2021-06-16T01:51:20Z" level=error msg="Unable to obtain ACME certificate for domains \"new3.example.com\": unable to generate a certificate for the domains [new3.example.com]: error: one or more domains had a problem:\n[new3.example.com] acme: error: 403 :: urn:ietf:params:acme:error:unauthorized :: Invalid response from http://new3.example.com/.well-known/acme-challenge/BUZWuWdNd2XAXwXCwkeqe5-PHb8cGV8V6UtzeLaKryE [2606:4700:3031::ac43:d1e9]: 404\n" providerName=myresolver.acme routerName=myapp@docker rule="Host(`new3.example.com`)"

Answer 1

You only need one docker-compose file for all the services, and there is no need to define one for each container.

The project structure you should be using should be something like:

├── docker-compose.yml
├── myapp
│   ├── .dockerignore
│   ├── Dockerfile
│   └── app
│       └── main.py
├── myapp1
│   ├── .dockerignore
│   ├── Dockerfile
│   └── app
│       └── main.py
└── traefik
    ├── acme.json
    └── traefik.yml

When creating containers, unless they are to be used for development purposes, it is recommended to not use a full-blown image, like ubuntu. Specifically for your purposes I would recommend a python image, such as python:3.7-slim.

Not sure if you are using this for development or production purposes, but you could also use volumes to mount the app directories inside the containers (especially useful if you are using this for development), and only use one Dockerfile for both myapp and myapp1, customizing it via environment variables.

Since you are already using traefik's dynamic configuration, I will do most of the setup for the container configuration via docker labels in the docker-compose.yml file.

Your dockerfile for myapp and myapp1 will be very similar at this point, but I've kept them as seperate ones, since you may need to make changes to them depending on the requirements of your apps in the future. I've used an environment variable for the port, which can allow you to change the port from your docker-compose.yml file.

You can use the following Dockerfile (./myapp/Dockerfile and ./myapp1/Dockerfile):

FROM python:3.7-slim

ARG DEBIAN_FRONTEND=noninteractive

ENV PYTHONUNBUFFERED=1

RUN pip3 install -U pip setuptools wheel && \
    pip3 install gunicorn fastapi uvloop httptools "uvicorn[standard]"

COPY . /app

ENV PORT=7777 # and 7778 for myapp1

ENTRYPOINT /usr/local/bin/gunicorn -b 0.0.0.0:$PORT -w 1 -k uvicorn.workers.UvicornWorker app.main:app --chdir /app

Note: you should really be using something like poetry or a requirements.txt file for your app dependencies.

The .dockerignore file (./myapp/.dockerignore and ./myapp1/.dockerignore) should contain:

Dockerfile

Since the whole directory is being copied inside the container and you don't need the Dockerfile to be in there.

Your main traefik config (./traefik/traefik.yml) can be something like:

providers:
  docker:
    exposedByDefault: false

global:
  checkNewVersion: false
  sendAnonymousUsage: false

api: {}
accessLog: {}

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: "websecure"
          scheme: "https"
  websecure:
    address: ":443"

ping:
  entryPoint: "websecure"

certificatesResolvers:
  myresolver:
    acme:
      caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
      email: "[email protected]"
      storage: "/etc/traefik/acme.json"
      httpChallenge:
        entryPoint: "web"

Note: The above acme config will use the stage letsencrypt server. Make sure all the details are correct, and remove caServer after you've tested that everything works, in order to communicate with the letsencrypt production server.

Your ./docker-compose.yml file should be something like:

version: "3.9"

services:
  myapp:
    build:
      context: ./myapp
      dockerfile: ./Dockerfile
    image: myapp
    depends_on:
      - traefik
    expose:
      - 7777
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.myapp.tls=true"
      - "traefik.http.routers.myapp.tls.certResolver=myresolver"
      - "traefik.http.routers.myapp.entrypoints=websecure"
      - "traefik.http.routers.myapp.rule=Host(`new2.example.com`)"
      - "traefik.http.services.myapp.loadbalancer.server.port=7777"
  myapp1:
    build:
      context: ./myapp1
      dockerfile: ./Dockerfile
    image: myapp1
    depends_on:
      - traefik
    expose:
      - 7778
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.myapp1.tls=true"
      - "traefik.http.routers.myapp1.tls.certResolver=myresolver"
      - "traefik.http.routers.myapp1.entrypoints=websecure"
      - "traefik.http.routers.myapp1.rule=Host(`new3.example.com`)"
      - "traefik.http.services.myapp1.loadbalancer.server.port=7778"
  traefik:
    image: traefik:v2.4
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./traefik/traefik.yml:/etc/traefik/traefik.yml
      - ./traefik/acme.json:/etc/traefik/acme.json
    ports:
      - 80:80
      - 443:443
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.tls=true"
      - "traefik.http.routers.api.tls.certResolver=myresolver"
      - "traefik.http.routers.api.entrypoints=websecure"
      - "traefik.http.routers.api.rule=Host(`new1.example.com`)"
      - "traefik.http.routers.api.service=api@internal"
      - "traefik.http.routers.api.middlewares=myAuth"
      - "traefik.http.middlewares.myAuth.basicAuth.users=admin:$$apr1$$4zjvsq3w$$fLCqJddLvrIZA.CCoGE2E." # generate with htpasswd. replace $ with $$

You can generate the password by using the command:

htpasswd -n admin | sed 's/\$/\$\$/g'

Note: If you need a literal dollar sign in the docker-compose file you need to use $$ as documented here.

Issuing docker-compose up in the directory should bring all the services up, and working as expected.

The above should work for you based on the details you have provided, but can be further improved at multiple points, depending on your needs.

Moreover, having the credentials for the traefik dashboard in the docker-compose.yml file is probably not the best, and you may want to use docker secrets for it. You can also add healthchecks and consider placing myapp and myapp1 into a seperate internal network.

If you want to get further into it, I propose that you start with Get started with Docker Compose and also read: Dockerfile reference and Compose file version 3 reference