How to modify a file in a Docker container when deploying with Terraform and Kubernetes?

Question

As a part of a bigger module, I want to deploy an nginx container and replace its default nginx.conf. The new config should be built using Terraform resources' data which is generated at the time of deployment. Is there a way to do it?

Answer 1

I managed to replace the standard nginx.conf with a dynamically generated one following these steps:

1. Create a template config file with placeholders for dynamic data
2. Parse the file using Terraform's template_file data source
3. Store the parsed data in a ConfigMap and mount the map as a volume for the Nginx container

Step by step:

Create nginx.conf template named nginx-conf.tpl:

events {
  worker_connections  4096;  ## Default: 1024
}
http {
  server {
    listen 80;
    listen [::]:80;

    server_name ${server_name};

    location /_plugin/kibana {
        proxy_pass https://${elasticsearch_kibana_endpoint};
    }
    location / {
        proxy_pass https://${elasticsearch_endpoint};
    }
  }
}

Parse the nginx-conf.tpl template with the following Terraform code:

data "template_file" "nginx" {
  template = "${file("${path.module}/nginx-conf.tpl")}"
  vars = {
    elasticsearch_endpoint        = "${aws_elasticsearch_domain.example-name.endpoint}"
    elasticsearch_kibana_endpoint = "${aws_elasticsearch_domain.example-name.kibana_endpoint}"
    server_name                   = "${var.server_name}"
  }
}

Create a ConfigMap and store the parsed template there with nginx.conf key:

resource "kubernetes_config_map" "nginx" {
  metadata {
    name = "nginx"
  }
  data = {
    "nginx.conf" = data.template_file.nginx.rendered
  }
}

Finally, mount the ConfigMap key as a container volume:

# ...
spec {
  # ...
  container {
    # ...
    volume_mount {
      name       = "nginx-conf"
      mount_path = "/etc/nginx"
    }
  }
  volume {
    name = "nginx-conf"
    config_map {
      name = "nginx"
      items {
        key  = "nginx.conf"
        path = "nginx.conf"
      }
    }
  }
}
# ...

That's it. Nginx server will start using the provided config.

Useful links: Kubernetes ConfigMap as volume, Terraform temple_file data source doc.