Mount a single config file onto an ECS service

Question

I come from a background in Kubernetes and I'm trying to learn AWS/ECS. In Kubernetes, you can use ConfigMap resources to mount simple one-off config files onto containers quickly and easily without having to go through all the trouble of setting up volumes. This also makes it very easy to configure services from Terraform, which is what I'm trying to do.

Do AWS ECS Services have a feature like the Kubernetes Config Maps? I just need the dead-simplest way to insert arbitrary text files into my services on startup, which can be updated with Terraform quickly. I want to avoid having to rebuild the whole image every time this file changes.

Is this possible or do I need to create volumes for this? If so, what's the best type of volume configuration for this purpose? I can store and update the files in S3 easily, and these are just simple config files that only need read access, so would this be an acceptable case to just mount the S3 bucket?

Answer 1

I've found a "best" solution, and fwiw - this problem vexed me for a year.

FIRST - there already an open feature request on the aws/containers-roadmap here: https://github.com/aws/containers-roadmap/issues/56

Please go to that github issue to show support. ECS could make this so much more idiomatic, easier, and intuitive.

Now, here is my approach:

First, ecs file composer uses a sidecar pattern to write files to a volume. https://github.com/compose-x/ecs-files-composer note: you don't necessarily need to use that ecs-files-composer sidecar mechanism, it doesn't matter how you get the files onto the volume.

BUT ASSUMING you do, then Setup the ECS file composer sidecar as a non-essential INIT container, and the essential container with a "dependsOn" criteria like this:

dependsOn: [
{ "containerName":"ecsFileComposerSidecar","condition":"COMPLETE" }
]

That pattern works well with terraform, you can create a module which accepts the parameters, with sane defaults, and a map of files you want to write, that reduces most of the boilerplate and outputs a sidecar container definition you can include in the ecs container definitions. That will make this pattern substantially more DRY (Don't repeat yourself). It also has the advantages of being able to use terraform for variable interpolation and/or injecting secrets into the environment of the ECS file composer init container using Jinja template syntax and the AWS secrets manager. All good stuff!

Okay, so now you've 🤞 hopefully got a ECS volume with your files on it.

Next, refer to this documentation: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bind-mounts.html

That suggests that a bind mount in ECS isn't a mount at all, it performs a file copy into the container. This solution presumes the file already exists, and you just want to overwrite it with a new copy. There are limitations and situations where this won't work though (for example when I try to mount a containerPath of /etc it won't even start the container and returns an obscure error)

If you want to write a new file that doesn't exist in the docker container, then you have two options:

1. make a copy of the docker image, add the file, publish to ECR, pull your version, then overwrite it using the copy/bindmount approach I just described.
2. mount the volume someplace else (i.e. /mnt/), and include a bash script that distributes the file/s -- then run your bash script with a command to copy the file. That description is here: https://kichik.com/2020/09/10/mounting-configuration-files-in-fargate/

Hopefully this all makes sense.
The second solution of write a bash script that runs, and using that as your container "command" seems to work all the time so that is the pattern I personally use.