Cannot access ports in AWS ECS EC2 instance

Question

I am running an AWS ECS service which is running a single task that has multiple containers. Tasks are run in awsvpc network mode. (EC2, not Fargate)

Container ports are mapped in the ECS task definition.

I added inbound rules in the EC2 Container instance security group (for ex: TCP 8883 -> access from anywhere). Also in the VPC network security group.

When I try to access the ports using Public IP of the instance from my remote PC, I get connection refused.

For ex: nc -z <PublicIP> <port>

When I SSH into the EC2 instance and try netstat, I can see SSH port 22 is listening, but not the container ports (ex: 8883). Also, when I do docker ps inside instance, Ports column is empty.

I could not figure out what configuration I missed. Kindly help.

PS: The destination (Public IP) is reachable from the remote PC. Just not from the port.

Answer 1

I am running an AWS ECS service which is running a single task that has multiple containers. Tasks are run in awsvpc network mode. (EC2, not Fargate)

Ec2, not Fargate, different horse for different courses. The task that is run against awsvpc network mode has own elastic network interface (ENI), a primary private IP address, and an internal DNS hostname. so how you will access that container with AWS EC2 public IP?

The task networking features provided by the awsvpc network mode give Amazon ECS tasks the same networking properties as Amazon EC2 instances. When you use the awsvpc network mode in your task definitions, every task that is launched from that task definition gets its own elastic network interface (ENI), a primary private IP address, and an internal DNS hostname. The task networking feature simplifies container networking and gives you more control over how containerized applications communicate with each other and other services within your VPCs.

task-networking

So you need to place LB and then configure your service behind LB.

when you create any target groups for these services, you must choose ip as the target type, not instance. This is because tasks that use the awsvpc network mode are associated with an ENI, not with an Amazon EC2 instance.

So something wrong with the configuration or lack of understanding between network mode. I will recommend reading this article.

when I do docker ps inside instance, Ports column is empty.

So it might be the case below if the port column is empty.

The host and awsvpc network modes offer the highest networking performance for containers because they use the Amazon EC2 network stack instead of the virtualized network stack provided by the bridge mode. With the host and awsvpc network modes, exposed container ports are mapped directly to the corresponding host port (for the host network mode) or the attached elastic network interface port (for the awsvpc network mode), so you cannot take advantage of dynamic host port mappings.

Keep the following in mind:

It’s available with the latest variant of the ECS-optimized AMI. It only affects creation of new container instances after opting into awsvpcTrunking. It only affects tasks created with awsvpc network mode and EC2 launch type. Tasks created with the AWS Fargate launch type always have a dedicated network interface, no matter how many you launch.

optimizing-amazon-ecs-task-density-using-awsvpc-network-mode