Amazon AWS ECS Docker Port not binding correctly

Question

I am using the ECS optimized ECS image and deploying using ECS.

So if i bash into the container and curl localhost i get the expected output (expected to be on port 80), this works fine.

Then if i run docker ps I get the following output

CONTAINER ID   IMAGE                              COMMAND             CREATED             STATUS              PORTS                        NAMES
1234           orgname/imagename:release-v0.3.1   "npm start"         53 minutes ago      Up 53 minutes       0.0.0.0:80->80/tcp           ecs-myname-1234`

Which would suggest port 80 is being mapped as expected. (I also see Amazon ECS Agent but have posted that above as not important)

Then i can run netstat -tulpn | grep :80 and i get the following output

(No info could be read for "-p": geteuid()=500 but you should be root.)
tcp        0      0 :::80                       :::*                        LISTEN      -  

Then as root i run sudo netstat -tulpn | grep :80 and i get the following output

tcp        0      0 :::80                       :::*                        LISTEN      21299/docker-proxy  

This makes me think it's only listening on the IPv6 interface? I as the host record for localhost is 127.0.0.1 that is why when i run curl localhost or curl 127.0.0.1 on the host i get curl: (56) Recv failure: Connection reset by peer

I have also checked the security groups and networks ACLS (not that they should have an effect on localhost)...

Any thoughts would be much appreciated!

Edit: For good measure (some people suggest netstat only shows ipv6 and not ipv4 when ipv6 is available. I have also ran this command lsof -OnP | grep LISTEN gives the following output

sshd       2360     root    3u     IPv4              10256       0t0        TCP *:22 (LISTEN)
sshd       2360     root    4u     IPv6              10258       0t0        TCP *:22 (LISTEN)
sendmail   2409     root    4u     IPv4              10356       0t0        TCP 127.0.0.1:25 (LISTEN)
exe        2909     root    4u     IPv4              13802       0t0        TCP 127.0.0.1:51678 (LISTEN)
exe       21299     root    4u     IPv6              68069       0t0        TCP *:80 (LISTEN)
exe       26395     root    4u     IPv6              89357       0t0        TCP *:8080 (LISTEN)

Answer 1

I ran into this exact problem when using the bridge network mode. I haven't found a solution yet. However, I have used two workarounds.

Workarounds

The easiest for me was to change NetworkMode to host in my ECS Task Definition.

Alternately, you can remove the need to know or care how ports are mapped by using an Application Load Balancer.

Network Modes

bridge maps the container port to another port (which may be different) on the host via the docker-proxy. This is the mode I have had trouble with in ECS.

host allows the container to open a port directly on the host, without requiring a proxy. The downside is that instances of the same container can't run on the same host without causing port conflicts.

awsvpc is like host, except it maps to an ENI in your VPC instead of a port on the host IP.

none is what it sounds like.

Application Load Balancer

Since posting this answer, the requirements of my project have changed. I haven't had a chance to go back and test port mappings in bridge mode directly. However, I am now using an Application Load Balancer to provide access to my containers.

When using an ALB, you don't have to worry about the host port at all. Instead, your ECS Service automatically adds your container as a target to a given ALB Target Group. This document goes over details on how to do that:

https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-load-balancing.html

Not elaborating more here because it's not a direct answer to the question about port binding issues.

---

Interestingly, network modes for ECS was announced just 5 days after you asked your question:

Announcement: https://aws.amazon.com/about-aws/whats-new/2016/08/amazon-ec2-container-service-now-supports-networking-modes-and-memory-reservation/

Network Mode Documentation: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RegisterTaskDefinition.html#ECS-RegisterTaskDefinition-request-networkMode

Hopefully this answer helps a few other Googlers. Note: I'll update this answer if I figure out how to get bridge mode working right in ECS.