Can I configure ECS to automatically purge old container [images] from my EC2 instances?

Question

I have what feels like a fairly standard ECS deployment:

• My cluster is backed by two EC2 instances, both running CoreOS with the ECS agent
• The cluster has a handful of services, each of which has associated tasks
• Each service is running one task at a time – usually the latest task definition
• The container images used in my task definitions are hosted in ECR

Several times now I’ve gone to deploy a new instance of a service, only to have ECS get unhappy because it’s unable to find space on my EC2 instance for the new image – I get “No space left on device” errors in CloudWatch.

If I SSH into the instance, I find that it’s got an old copy of all my Docker images and containers – almost 100 of the former, 500 of the latter. I can free up disk space by running the docker rm and docker rmi commands manually, but I naively assumed this was the sort of thing ECS would manage for me.

When I’m setting up services and tasks, I never usually interact with the individual hosts or containers – I trust ECS to schedule containers for me. I expected it would also purge old container images to free up disk space as required. (I’m aware this isn’t functionality provided by Docker.)

I could set alarms to monitor disk space on my EC2 instances, and log on to free disk space as required – but I’m wondering if I’ve missed something.

Can I tell ECS to purge old/unused container [images] from my EC2 instances?

Answer 1

Yes! Since version 1.13.0, the ECS agent has automatically cleaned up old, unused images. You can see the default settings and find how to customize them in the README (check out ECS_IMAGE_CLEANUP_INTERVAL, ECS_IMAGE_MINIMUM_CLEANUP_AGE, and ECS_NUM_IMAGES_DELETE_PER_CYCLE).