Docker build hangs. How can I see what is going on?

Question

This Dockerfile hangs after the download has completed:

FROM ubuntu:18.04
MAINTAINER Dean Schulze

ENV JS_CE_VERSION 7.1.0
ENV JS_CE_HOME /opt/jasperreports-server-cp-${JS_CE_VERSION}
ENV PATH $PATH:${JS_CE_HOME}

RUN apt-get update && apt-get install -y wget \
&& wget --progress=bar:force:noscroll -O TIB_js-jrs-cp_${JS_CE_VERSION}_linux_x86_64.run https://sourceforge.net/projects/jasperserver/files/JasperServer/JasperReports%20Server%20Community%20Edition%20${JS_CE_VERSION}/TIB_js-jrs-cp_${JS_CE_VERSION}_linux_x86_64.run \
&& chmod a+x TIB_js-jrs-cp_${JS_CE_VERSION}_linux_x86_64.run \
&& /TIB_js-jrs-cp_${JS_CE_VERSION}_linux_x86_64.run --mode unattended --jasperLicenseAccepted yes --postgres_password Postgres1 --prefix ${JS_CE_HOME} \
&& rm TIB_js-jrs-cp_${JS_CE_VERSION}_linux_x86_64.run \
&& rm -rf ${JS_CE_HOME}/apache-ant ${JS_CE_HOME}/buildomatic \
${JS_CE_HOME}/docs ${JS_CE_HOME}/samples ${JS_CE_HOME}/scripts \
&& apt-get clean

EXPOSE 8080

CMD ctlscript.sh start && tail -f /dev/null

The last few lines of output are

Resolving superb-dca2.dl.sourceforge.net (superb-dca2.dl.sourceforge.net)... 209.61.193.20
Connecting to superb-dca2.dl.sourceforge.net (superb-dca2.dl.sourceforge.net)|209.61.193.20|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 343517555 (328M) [application/x-makeself]
Saving to: 'TIB_js-jrs-cp_7.1.0_linux_x86_64.run'

TIB_js-jrs-cp_7.1.0 100%[===================>] 327.60M  1.60MB/s    in 3m 52s  

2018-07-28 03:15:28 (1.41 MB/s) - 'TIB_js-jrs-cp_7.1.0_linux_x86_64.run' saved [343517555/343517555]

How do I diagnose a docker build that hangs like this?

Edit

Here's the requested output:

$ docker image history d5d47e51eafc
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
d5d47e51eafc        23 minutes ago      /bin/sh -c #(nop)  ENV PATH=/usr/local/sbin:…   0B                  
831a3a551fa7        23 minutes ago      /bin/sh -c #(nop)  ENV JS_CE_HOME=/opt/jaspe…   0B                  
e8361426e492        23 minutes ago      /bin/sh -c #(nop)  ENV JS_CE_VERSION=6.3.0      0B                  
7af364f52b1b        23 minutes ago      /bin/sh -c #(nop)  MAINTAINER JS Minet          0B                  
735f80812f90        30 hours ago        /bin/sh -c #(nop)  CMD ["/bin/bash"]            0B                  
<missing>           30 hours ago        /bin/sh -c mkdir -p /run/systemd && echo 'do…   7B                  
<missing>           30 hours ago        /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$…   2.76kB              
<missing>           30 hours ago        /bin/sh -c rm -rf /var/lib/apt/lists/*          0B                  
<missing>           30 hours ago        /bin/sh -c set -xe   && echo '#!/bin/sh' > /…   745B                
<missing>           30 hours ago        /bin/sh -c #(nop) ADD file:4bb62bb0587406855…   83.5MB              

It looks like the lines are in reverse order and the last one executed is the ENV PATH. The next line would be the RUN line which has multiple commands separated by &&.

This is a modification of a Dockerfile on DockerHub. I changed ubuntu versions and the version of the app being installed. That shouldn't have broken anything.

Should I file a bug report?

Answer 1

First list the "layers" of your finished or incomplete image. Each layer typically corresponds to an instruction in your Dockerfile.

Identify the image ID using

docker image ls

then list the layers inside the image using

docker image history <ID>

You will see something like this:

IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
6c32fe3da793        2 days ago          /bin/sh -c #(nop) COPY file:c25ef1dcc737cb59…   635B                
4c1309db9b9c        2 days ago          /bin/sh -c #(nop) COPY dir:30506cf0fc0cdb096…   8.64kB              
5f5ae40b5fd5        3 weeks ago         /bin/sh -c apk update && apk add --no-cache …   164MB               
989d78741d0b        3 weeks ago         /bin/sh -c #(nop)  ENV DOCKER_VERSION=18.03.…   0B                  
6160911711fc        3 weeks ago         /bin/sh -c #(nop)  CMD ["python3"]              0B                  
... etc

Then create a container from any point inside your image. From there you can perform the next instruction that would cause a problem in your Dockerfile.

Eg:

docker run -it --rm 4c1309db9b9c sh

Answer 2

At some point in the last couple of years, Buildkit has become the default Docker backend. Buildkit does not write out intermediate layers as images, as a performance optimization. Therefore, if you need to debug a hanging docker build, comment out the line that the build is hanging at and all subsequent lines. You now have a Dockerfile that you can execute $ docker build . with. Once the build is finished, you can launch the image, bash into the container, run the command that is causing the build to hang and then inspect the state of the container to debug the situation.