Docker ubuntu build fails on ec2

Question

Endevouring to learn more about Docker. Want to do my work on an ec2 instance. Thought this tutorial was a good place to start: Docker Basics. When I try to build the docker image from the Dockerfile I get errors when running apt-get update -y.

Initially thought an unsupported ubuntu release (12.04) was an issue so updated that reference to latest (17.04) and still errors relating to apt-update (below).

Update: I ran the steps on a non-ec2 machine and got a different output at apt-update stage, so something in ec2 configuration is I think interfering with network connectivity.

Any help appreciated. Console output below:

ec2 docker build --no-cache -t moikaturns/amazon-ecs-sample .
Sending build context to Docker daemon   363 kB
Step 1/12 : FROM ubuntu:17.04
 ---> bde41be8de8c
Step 2/12 : RUN apt-get update -y
 ---> Running in d8132cda2e2a
Err:1 http://security.ubuntu.com/ubuntu zesty-security InRelease
  Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1560:8001::14). - connect (101: Network is unreachable) [IP: 2001:67c:1560:8001::14 80]
Err:2 http://archive.ubuntu.com/ubuntu zesty InRelease
  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::21). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8001::21 80]
Err:3 http://archive.ubuntu.com/ubuntu zesty-updates InRelease
  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::21). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8001::21 80]
Err:4 http://archive.ubuntu.com/ubuntu zesty-backports InRelease
  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::21). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8001::21 80]
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/zesty/InRelease  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::21). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8001::21 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/zesty-updates/InRelease  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::21). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8001::21 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/zesty-backports/InRelease  Cannot initiate the connection to archive.ubuntu.com:80 (2001:67c:1360:8001::21). - connect (101: Network is unreachable) [IP: 2001:67c:1360:8001::21 80]
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/zesty-security/InRelease  Cannot initiate the connection to security.ubuntu.com:80 (2001:67c:1560:8001::14). - connect (101: Network is unreachable) [IP: 2001:67c:1560:8001::14 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.
 ---> 6aa041d69aec
Removing intermediate container d8132cda2e2a
Step 3/12 : RUN apt-get install -y git curl apache2 php5 libapache2-mod-php5 php5-mcrypt php5-mysql
 ---> Running in 1b807c9fd593
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package git
E: Unable to locate package curl
E: Unable to locate package apache2
E: Unable to locate package php5
E: Unable to locate package libapache2-mod-php5
E: Unable to locate package php5-mcrypt
E: Unable to locate package php5-mysql
The command '/bin/sh -c apt-get install -y git curl apache2 php5 libapache2-mod-php5 php5-mcrypt php5-mysql' returned a non-zero code: 100

Contents of /etc/resolve.conf obtained when the Dockerbuild script runs. It is system generated and sourced from the host machine (to which I added the google DNS). The output from the Dockerfile script confirms the contents in the context of the script running:

Step 2/13 : RUN cat /etc/resolv.conf
 ---> Running in 68b494d1fcdb
; generated by /sbin/dhclient-script
search eu-west-1.compute.internal
nameserver 172.31.0.2
nameserver 8.8.8.8

Running this on the ec2 instance works but I cannot run it inside the Dockerfile script (because the ubuntu does not come with curl installed and I cannot run apt-get to install anything due to the network issue):

ec2 curl ipinfo.io
{
  "ip": "52.17.159.7",
  "hostname": "ec2-52-17-159-7.eu-west-1.compute.amazonaws.com",
  "city": "Dublin",
  "region": "Leinster",
  "country": "IE",
  "loc": "53.3389,-6.2595",
  "org": "AS16509 Amazon.com, Inc."
}

Answer 1

Discovered how to get it working by specifying a different network for docker to use/bind to (an alternative named 'host' in contrast to what I believe is the default option 'bridge'). I cannot explain why one worked and the other not as my networking knowledge is not great.

This article prompted me to try it Docker container networking. It describes three 'Default networks' created by docker. When invoking docker it's possible to select a specific network to bind to. I assumed 'none' literally means no network. I tried 'bridge' and the same symptoms presented. I tried 'host' and internet connectivity was achieved (apt-get update worked), ergo that answers my question.

Output of the command to list the networks available to docker:

ec2 docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
4d37091ae99e        bridge              bridge              local
cab1b88f8084        host                host                local
592bd8b26a6a        none                null                local

Specifying a network when running docker build (the --network switch is the key), the output of apt-get update is my evidence the change works:

ec2 docker build --network host --no-cache -t moikaturns/amazon-ecs-sample .
Sending build context to Docker daemon 363.5 kB
Step 1/3 : FROM ubuntu:17.04
 ---> bde41be8de8c
Step 2/3 : RUN cat /etc/resolv.conf
 ---> Running in 62447e6613d6
; generated by /sbin/dhclient-script
search eu-west-1.compute.internal
nameserver 172.31.0.2
 ---> 7456dd87e8eb
Removing intermediate container 62447e6613d6
Step 3/3 : RUN apt-get update -y
 ---> Running in 19c5ba6e2a21
Get:1 http://security.ubuntu.com/ubuntu zesty-security InRelease [89.2 kB]
Get:2 http://archive.ubuntu.com/ubuntu zesty InRelease [243 kB]
Get:3 http://archive.ubuntu.com/ubuntu zesty-updates InRelease [89.2 kB]
...
Get:21 http://archive.ubuntu.com/ubuntu zesty-backports/main amd64 Packages [1438 B]
Fetched 24.2 MB in 3s (7702 kB/s)
Reading package lists...
 ---> bcf0a72ac69e
Removing intermediate container 19c5ba6e2a21
Successfully built bcf0a72ac69e

Also found this useful from Amazon Web Services (AWS) EC2 example:

docker run -it ubuntu bash

Tells docker to launch an interactive bash shell. Network difficulties can be observed interactively rather than running the Dockerfile script. The frustrating thing is it's a minimal ubuntu and basic tools like ping are not installed. Since apt-get install is not working it's possible to install them to try do much investigation.

Specifying the network that works with docker run enables network functionality:

docker run --network=host -it ubuntu bash

Thanks to all whom responded to my question.

RCA: after following this up with aws a representative kindly assisted in diagnosing what was wrong (the tutorial works fine on a vanilla ec2 instance). Cause was traced to an iptables rule I'd added to my ec2 instance:

sudo /sbin/iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 8080

It redirects inbound port 80 to internal port 8080. Clearly it had upset iptables entries created by Docker(somehow). If I remove this rule the vanilla tutorial works on my ec2 instance (all the iptables rule was doing was routing port 80 to 8080, which I can do using a proxy such as nginx instead).