Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Docker push seems not to update image => Layer already exists

Tags:

docker

I'm building an elasticsearch image with Dockerfile:

FROM alpine:latest

RUN apk update \
    && apk upgrade \
    && apk add curl wget bash openssl openjdk8 \
    && rm -rf /var/cache/apk/*


WORKDIR /root/

RUN wget -e use_proxy=yes https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.1.tar.gz -O elasticsearch-6.1.1.tar.gz

RUN tar -xf  elasticsearch-6.1.*.tar.gz -C /usr/local/ \
    && mv /usr/local/elasticsearch-6.1* /usr/local/elasticsearch \
    && mkdir /usr/local/elasticsearch/logs \
    && mkdir /usr/local/elasticsearch/data \
    && echo '-Xms512m' > /usr/local/elasticsearch/config/jvm.options \
    && echo '-Xmx512m' >> /usr/local/elasticsearch/config/jvm.options \
    && adduser -D -u 1000 -h /usr/local/elasticsearch elasticsearch \
    && chown -R elasticsearch /usr/local/elasticsearch

USER elasticsearch

CMD ["/usr/local/elasticsearch/bin/elasticsearch", "-Ecluster.name=es-cluster", "-Enode.name=${HOSTNAME}", "-Epath.data=/usr/local/elasticsearch/data", "-Epath.logs=/usr/local/elasticsearch/logs", "-Enetwork.host=0.0.0.0", "-Ediscovery.zen.ping.unicast.hosts=es-master", "-Expack.security.enabled=false", "-Expack.monitoring.enabled=false"]

Then I build it

docker build -t 'es:6.1' .

I tag it and push it to registry:

docker login
docker tag es:6.1 <username>/es:6.1
docker push <username>es:6.1

This works well, but I when I make a change to Dockerfile, and do the whole process again ( without changing tag ), I get:

docker push <user>/es:6.1      
The push refers to repository [docker.io/<user>/es]
1db25bdc763a: Layer already exists 
3bf058fc2f20: Layer already exists 
5570ea7cd957: Layer already exists 
cd7100a72410: Layer already exists 
6.1: digest: sha256:f55a86abbb2593299985d0c0a5de8be69eb0b056d664b0e7d020e63fae0d7d82 size: 1164

That seems to indicate image were not updated...

What am I missing ? It seems to be a quite basic operation in the Docker workflow...

EDIT:

After deleting all images locally and remotely, I did a build again:

docker build -t 'es:latest' .

I know use latest tag, as it is closer to my use case

Here is the output

docker build -t es:latest .
Sending build context to Docker daemon  1.051MB
Step 1/12 : FROM alpine:latest
 ---> 3fd9065eaf02
Step 2/12 : ENV http_proxy=http://10.xxx.xxx.65:8080
 ---> Running in 5d2583ff56e2
Removing intermediate container 5d2583ff56e2
 ---> d4a09d1a4034
Step 3/12 : ENV https_proxy=http://10.xxx.xxx.65:8080
 ---> Running in 785bb47b903e
Removing intermediate container 785bb47b903e
 ---> 3412535b08f9
Step 4/12 : ENV HTTP_PROXY=http://10.xxx.xxx.65:8080
 ---> Running in ed77d05a070c
Removing intermediate container ed77d05a070c
 ---> b7e5304c4f6b
Step 5/12 : ENV HTTPS_PROXY=http://10.xxx.xxx.65:8080
 ---> Running in 197eb0ba578f
Removing intermediate container 197eb0ba578f
 ---> 3609a88fead7
Step 6/12 : RUN apk update     && apk upgrade     && apk add curl wget bash openssl openjdk8     && rm -rf /var/cache/apk/*
 ---> Running in fe0a8813e122
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
v3.7.0-50-gc8da5122a4 [http://dl-cdn.alpinelinux.org/alpine/v3.7/main]
v3.7.0-49-g06d6ae04c3 [http://dl-cdn.alpinelinux.org/alpine/v3.7/community]
OK: 9044 distinct packages available
OK: 4 MiB in 11 packages
(1/53) Installing pkgconf (1.3.10-r0)
(2/53) Installing ncurses-terminfo-base (6.0_p20170930-r0)
(49/53) Installing openjdk8 (8.151.12-r0)
    (50/53) Installing libcrypto1.0 (1.0.2n-r0)
(51/53) Installing libssl1.0 (1.0.2n-r0)
(52/53) Installing openssl (1.0.2n-r0)
(53/53) Installing wget (1.19.2-r1)
Executing busybox-1.27.2-r7.trigger
Executing ca-certificates-20171114-r0.trigger
Executing java-common-0.1-r0.trigger
OK: 113 MiB in 64 packages
Removing intermediate container fe0a8813e122
 ---> 7cb92f7437a6
Step 7/12 : WORKDIR /root/
Removing intermediate container e550c56108de
 ---> 585c1062a9b0
Step 8/12 : RUN wget -e use_proxy=yes https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.1.tar.gz -O elasticsearch-6.1.1.tar.gz
 ---> Running in 3b744a9cc35f
--2018-01-10 14:45:51--  https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.1.tar.gz
Connecting to 10.31.255.65:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 28462503 (27M) [application/x-gzip]
Saving to: 'elasticsearch-6.1.1.tar.gz'

     0K .......... .......... .......... .......... ..........  0%  135K 3m26s
    50K .......... .......... .......... .......... ..........  0%  178K 3m0s
   100K .......... .......... .......... .......... ..........  0%  272K 2m34s
   150K .......... .......... .......... .......... ..........  0%  530K 2m8s
   200K .......... .......... .......... .......... ..........  0%  528K 1m53s
   250K .......... .......... .......... .......... ..........  1%  557K 1m42s
   300K .......... .......... .......... .......... ..........  1% 10.6M 88s
   350K .......... .......... .......... .......... ..........  1%  538K 83s
   400K .......... .......... .......... .......... ..........  1%  545K 79s
 27650K .......... .......... .......... .......... .......... 99% 5.24M 0s
 27700K .......... .......... .......... .......... .......... 99%  645K 0s
 27750K .......... .......... .......... .......... .....     100% 6.80M=22s

2018-01-10 14:46:13 (1.25 MB/s) - 'elasticsearch-6.1.1.tar.gz' saved [28462503/28462503]

Removing intermediate container 3b744a9cc35f
 ---> 6f987b31a830
Step 9/12 : RUN tar -xf  elasticsearch-6.1.*.tar.gz -C /usr/local/     && mv /usr/local/elasticsearch-6.1* /usr/local/elasticsearch     && mkdir /usr/local/elasticsearch/logs     && mkdir /usr/local/elasticsearch/data     && echo '-Xms512m' > /usr/local/elasticsearch/config/jvm.options     && echo '-Xmx512m' >> /usr/local/elasticsearch/config/jvm.options     && adduser -D -u 1000 -h /usr/local/elasticsearch elasticsearch     && chown -R elasticsearch /usr/local/elasticsearch
 ---> Running in d8ee61eb83ab
Removing intermediate container d8ee61eb83ab
 ---> 6aa4b9c19b82
Step 10/12 : USER root
 ---> Running in 46ffe8e22d78
Removing intermediate container 46ffe8e22d78
 ---> c6f71d927725
Step 11/12 : CMD ["/usr/local/elasticsearch/bin/elasticsearch", "-Ecluster.name=es-cluster", "-Enode.name=${HOSTNAME}", "-Epath.data=/usr/local/elasticsearch/data", "-Epath.logs=/usr/local/elasticsearch/logs", "-Enetwork.host=0.0.0.0", "-Ediscovery.zen.ping.unicast.hosts=es-master", "-Expack.security.enabled=false", "-Expack.monitoring.enabled=false"]
 ---> Running in 65240d2a060a
Removing intermediate container 65240d2a060a
 ---> 90b6c90b1451
Step 12/12 : EXPOSE 9200 9300
 ---> Running in 1aa75c0dfcef
Removing intermediate container 1aa75c0dfcef
 ---> 7ad4b97d341f
Successfully built 7ad4b97d341f
Successfully tagged es:latest

Now I have an image:

es                                                         latest               7ad4b97d341f        2 minutes ago       171MB

I tag it and push it to registry:

docker tag es:latest <username>/es:latest
docker push <username>es:latest

Here, I can see the push actually pushing.

docker push xoco70/es:latest
The push refers to repository [docker.io/xoco70/es]
0a7ea1f8eaee: Pushed 
a9a17f2ab87e: Pushed 
b574c33babc4: Pushed 
cd7100a72410: Layer already exists 
latest: digest: sha256:e73b618862362f8df013b7806317142fef4384a64066f0488d59c778349590ab size: 1164

Now I have 2 images, es and xoco70/es

Now second part, I just change user from root to elasticsearch on Dockerfile

I do it again:

docker build -t es:latest .          
  Sending build context to Docker daemon  1.051MB
  Step 1/12 : FROM alpine:latest
   ---> 3fd9065eaf02
  Step 2/12 : ENV http_proxy=http://10.31.255.65:8080
   ---> Using cache
   ---> d4a09d1a4034
  Step 3/12 : ENV https_proxy=http://10.31.255.65:8080
   ---> Using cache
   ---> 3412535b08f9
  Step 4/12 : ENV HTTP_PROXY=http://10.31.255.65:8080
   ---> Using cache
   ---> b7e5304c4f6b
  Step 5/12 : ENV HTTPS_PROXY=http://10.31.255.65:8080
   ---> Using cache
   ---> 3609a88fead7
  Step 6/12 : RUN apk update     && apk upgrade     && apk add curl wget bash openssl openjdk8     && rm -rf /var/cache/apk/*
   ---> Using cache
   ---> 7cb92f7437a6
  Step 7/12 : WORKDIR /root/
   ---> Using cache
   ---> 585c1062a9b0
  Step 8/12 : RUN wget -e use_proxy=yes https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.1.tar.gz -O elasticsearch-6.1.1.tar.gz
   ---> Using cache
   ---> 6f987b31a830
  Step 9/12 : RUN tar -xf  elasticsearch-6.1.*.tar.gz -C /usr/local/     && mv /usr/local/elasticsearch-6.1* /usr/local/elasticsearch     && mkdir /usr/local/elasticsearch/logs     && mkdir /usr/local/elasticsearch/data     && echo '-Xms512m' > /usr/local/elasticsearch/config/jvm.options     && echo '-Xmx512m' >> /usr/local/elasticsearch/config/jvm.options     && adduser -D -u 1000 -h /usr/local/elasticsearch elasticsearch     && chown -R elasticsearch /usr/local/elasticsearch
   ---> Using cache
   ---> 6aa4b9c19b82
  Step 10/12 : USER elasticsearch
   ---> Running in dbed466ac2de
  Removing intermediate container dbed466ac2de
   ---> 461798559927
  Step 11/12 : CMD ["/usr/local/elasticsearch/bin/elasticsearch", "-Ecluster.name=es-cluster", "-Enode.name=${HOSTNAME}", "-Epath.data=/usr/local/elasticsearch/data", "-Epath.logs=/usr/local/elasticsearch/logs", "-Enetwork.host=0.0.0.0", "-Ediscovery.zen.ping.unicast.hosts=es-master", "-Expack.security.enabled=false", "-Expack.monitoring.enabled=false"]
   ---> Running in 553c6fc0e646
  Removing intermediate container 553c6fc0e646
   ---> a78743b5e61f
  Step 12/12 : EXPOSE 9200 9300
   ---> Running in 505c3dfd09f0
  Removing intermediate container 505c3dfd09f0
   ---> 6c318f659633
  Successfully built 6c318f659633
  Successfully tagged es:latest

I tag, and push

docker tag es:latest xoco70/es:latest
docker push xoco70/es:latest         
The push refers to repository [docker.io/xoco70/es]
0a7ea1f8eaee: Layer already exists 
a9a17f2ab87e: Layer already exists 
b574c33babc4: Layer already exists 
cd7100a72410: Layer already exists 
latest: digest: sha256:c76d5a0be0160832a198ca591a36f993b68222278a05085b8dd0dfacd27ca3fa size: 1164

When I check with docker history:

sudo docker pull xoco70/es                     
  Using default tag: latest
  latest: Pulling from xoco70/es
  Digest: sha256:c76d5a0be0160832a198ca591a36f993b68222278a05085b8dd0dfacd27ca3fa
  Status: Image is up to date for xoco70/es:latest

    NT
    sha256:6c318f65963364ad5266218122883f5a11cb36fa62cf9b3715c8774de866b030   3 minutes ago       /bin/sh -c #(nop)  EXPOSE 9200 9300                                                                                                                                                                                                                                                                                                                                                                                                                                                          0B                  
    sha256:a78743b5e61f29d3562cfc9db85e6c3d4b7f8c05314e223c75e4638cc61f834a   3 minutes ago       /bin/sh -c #(nop)  CMD ["/usr/local/elasticsearch/bin/elasticsearch" "-Ecluster.name=es-cluster" "-Enode.name=${HOSTNAME}" "-Epath.data=/usr/local/elasticsearch/data" "-Epath.logs=/usr/local/elasticsearch/logs" "-Enetwork.host=0.0.0.0" "-Ediscovery.zen.ping.unicast.hosts=es-master" "-Expack.security.enabled=false" "-Expack.monitoring.enabled=false"]                                                                                                                              0B                  
    sha256:461798559927fd64b89cd542fc0c4a44a705701d03db371039e6d5dec5fe82f9   3 minutes ago       /bin/sh -c #(nop)  USER elasticsearch                                                                                                                                                                                                                                                                                                                                                                                                                                                        0B                  
    sha256:6aa4b9c19b82fb3eb5c3f1e724d4c19bc5cefd1a181d6e9f48f458c038eb452a   9 minutes ago       /bin/sh -c tar -xf  elasticsearch-6.1.*.tar.gz -C /usr/local/     && mv /usr/local/elasticsearch-6.1* /usr/local/elasticsearch     && mkdir /usr/local/elasticsearch/logs     && mkdir /usr/local/elasticsearch/data     && echo '-Xms512m' > /usr/local/elasticsearch/config/jvm.options     && echo '-Xmx512m' >> /usr/local/elasticsearch/config/jvm.options     && adduser -D -u 1000 -h /usr/local/elasticsearch elasticsearch     && chown -R elasticsearch /usr/local/elasticsearch   31.9MB              
    sha256:6f987b31a830723196f13d0caf5893af5d8d5f507508149a68b42d8c503ef7ec   9 minutes ago       /bin/sh -c wget -e use_proxy=yes https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.1.tar.gz -O elasticsearch-6.1.1.tar.gz                                                                                                                                                                                                                                                                                                                                               28.5MB              
    sha256:585c1062a9b0fc66624a6e5ccd7e9a1d16fdfa47558289cc66eee5d553db5af0   9 minutes ago       /bin/sh -c #(nop) WORKDIR /root/                                                                                                                                                                                                                                                                                                                                                                                                                                                             0B                  
    sha256:7cb92f7437a6420234bda929407b69b2f71496fd769af75c7d74c9d4a7055a64   9 minutes ago       /bin/sh -c apk update     && apk upgrade     && apk add curl wget bash openssl openjdk8     && rm -rf /var/cache/apk/*                                                                                                                                                                                                                                                                                                                                                                       106MB               
    sha256:3609a88fead7b3402a0bbca21c465d402220f002cac95cd7d069ae5abfdf33f0   12 minutes ago      /bin/sh -c #(nop)  ENV HTTPS_PROXY=http://10.31.255.65:8080                                                                                                                                                                                                                                                                                                                                                                                                                                  0B                  
    sha256:b7e5304c4f6b9d5a277dc73af37556033e1f1552e61739de9c42ed26d05085e2   12 minutes ago      /bin/sh -c #(nop)  ENV HTTP_PROXY=http://10.31.255.65:8080                                                                                                                                                                                                                                                                                                                                                                                                                                   0B                  
    sha256:3412535b08f948c8a65a2291dfaf7f0d018b464307bc42b95c6b284ccead6d64   12 minutes ago      /bin/sh -c #(nop)  ENV https_proxy=http://10.31.255.65:8080                                                                                                                                                                                                                                                                                                                                                                                                                                  0B                  
    sha256:d4a09d1a4034f0c734b5610a420387d911041d91d9a15d1a85eeb83a2c7ff5ce   12 minutes ago      /bin/sh -c #(nop)  ENV http_proxy=http://10.31.255.65:8080                                                                                                                                                                                                                                                                                                                                                                                                                                   0B                  
    sha256:3fd9065eaf02feaf94d68376da52541925650b81698c53c6824d92ff63f98353   18 hours ago        /bin/sh -c #(nop)  CMD ["/bin/sh"]                                                                                                                                                                                                                                                                                                                                                                                                                                                           0B                  
    <missing>                                                                 18 hours ago        /bin/sh -c #(nop) ADD file:093f0723fa46f6cdbd6f7bd146448bb70ecce54254c35701feeceb956414622f in /                                                                                                                                                                                                                                                                                                                                                                                             4.14MB 

So it appears to be taking my changes even if it doesn't seem. It is weird, because I already did that before, and it was not working, maybe we have to wait a bit so the registry update his data...

like image 645
Juliatzin Avatar asked Jan 10 '18 13:01

Juliatzin


1 Answers

without changing tag

When you rebuild the image using docker build -t 'es:6.1' . you als0 need to retag this new image, since the image <user>/es:6.1 will still refer to the old es:6.1

So whenever you do docker build -t 'es:6.1' . you also need to run docker tag es:6.1 <username>/es:6.1

Update

It might possible (though unusual) that the change you have done to the dockerfile was previously uploaded to the registry.

To be 100% sure, pull the image back from the registry and inspect Dockerfile using docker history --no-trunc <user>/es:6.1>.

If you can spot the change in the dockerfile that you have made, then the push was successful.

like image 75
yamenk Avatar answered Oct 12 '22 20:10

yamenk