docker build using previous build caches from registry

Question

I'm configuring a bamboo build plan to build docker images. Using AWS ECS as registry. Build plan is something like this;

• pull the latest tag

  docker pull xxx.dkr.ecr.eu-west-1.amazonaws.com/myimage:latest
  

• build image with latest tag

  docker build -t myimage:latest .
  

• tag the image (necessary for ECS)

  docker tag -f myimage:latest xxx.dkr.ecr.eu-west-1.amazonaws.com/myimage:latest
  

• Push the image to the registry

  docker push xx.dkr.ecr.eu-west-1.amazonaws.com/myimage:latest
  

Because build tasks run on different and fresh build engines/servers every time, It doesn't have local cache.

When I don't change anything at Dockerfile and execute it again(at another server), I would expect docker to use local cache(comes from docker pull) and doesn't execute each line again. But it tries to build image everytime. I was also expecting that when I change something at the bottom of the file, it's going to use cache and executes only the latest line, but I'm not sure about this.

Do I know something wrong or are there any opinions on approach?

Answer 1

are you considering using squid proxy ?

edit : in case you dont want to go to the official website above, here is quick setup on squid proxy (debian based)

apt-get install squid-deb-proxy

and then change the squid configuration to create a larger space by open up

/etc/squid/squid.conf

and replace #cache_dir ufs /var/spool/squid with cache_dir ufs /var/spool/ squid 10000 16 256 and there you go,, a 10.000 MB worth of cache space

and then point the proxy address in dockerfile,, here is a example of dockerfile with squid proxy

yum and apt-get based distro:

apt-get based distro

`FROM debian
RUN apt-get update -y && apt-get install net-tools
RUN echo "Acquire::http::Proxy \"http://$( \
route -n | awk '/^0.0.0.0/ {print $2}' \
):8000\";" \ > /etc/apt/apt.conf.d/30proxy
RUN echo "Acquire::http::Proxy::ppa.launchpad.net DIRECT;" >> \
/etc/apt/apt.conf.d/30proxy
CMD ["/bin/bash"]`

yum based distro

`FROM centos:centos7
RUN yum update -y && yum install -y net-tools
RUN echo "proxy=http://$(route -n | \
awk '/^0.0.0.0/ {print $2}'):3128" >> /etc/yum.conf
RUN sed -i 's/^mirrorlist/#mirrorlist/' \
/etc/yum.repos.d/CentOS-Base.repo
RUN sed -i 's/^#baseurl/baseurl/' \
/etc/yum.repos.d/CentOS-Base.repo
RUN rm -f /etc/yum/pluginconf.d/fastestmirror.conf
RUN yum update -y
CMD ["/bin/bash"]`

lets say you install squid proxy in your aws registry,,only the first build would fetch the data from the internet the rest (another server) build should be from the squid proxy cached. .

this technique based on book docker in practice technique 57 with tittle set up a package cache for faster build

i dont think there is a cache feature in docker without any third party software.. maybe there is and i just dont know it. .im not sure,, just correct me if im wrong. .