Amazon AWS NAT Gatway not working, EC2 doesn't register in ECS Cluster

Question

I'm convinced that my EC2 instance and cluster are configured correctly and that they won't register due to the network configuration.

I am building EC2 instance from a auto-scaling group, they will register if I create/associate an Elastic IP to them however I cannot get them to NAT, as they will be auto-scaled I need them to use NAT to register themselves with the cluster.

I have 1 VPC on network 10.0.0.0/0

I have 3 subnets in 3 regions on ranges 10.0.0.0/24, 10.0.1.0/24 & 10.0.2.0/24.

I want to use Amazon's NAT Gateway not to be confused with NAT Instance which is an AMI to do this functionality.

I have been following this guide @ http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-nat-gateway.html

(For now) I have a single route table (which all 3 subnets are assigned to) which is as follows...

10.0.0.0/16 | local
0.0.0.0/0   | nat-******

The NAT Gateway sits on 10.0.2.19.

The security group is as follows

Inbound
ALL TRAFFIC | ALL PROTOCOLS | ALL PORT RANGE |SOURCE: 0.0.0.0/0
Outbound
ALL TRAFFIC | ALL PROTOCOLS | ALL PORT RANGE |DESTINATION: 0.0.0.0/0

Network ACL is

Inbound
Rule # | Type | Protocol | Port Range | Source | Allow / Deny
100 | ALL Traffic | ALL | ALL | 0.0.0.0/0 | ALLOW
* | ALL Traffic | ALL | ALL | 0.0.0.0/0 | DENY
Outbound
Rule # | Type | Protocol | Port Range | Destination | Allow / Deny
100 | ALL Traffic | ALL | ALL | 0.0.0.0/0 | ALLOW
* | ALL Traffic | ALL | ALL | 0.0.0.0/0 | DENY

Which suggests everything is allowed?

I have attached a IP to one instance then connected over the VPC to another instance and try to ping the NAT gateway which is unsuccessful but I don't understand why? Everything here is set quite simply and it doesn't quite make sense?

Help much appreciated :)

Answer 1

You have all 3 subnets as private. The NAT gateway has to be in public subnet and its routing table should have an entry 0.0.0.0/0 which should route to the internet gateway.

How do you expect the NAT gateway to route internet traffic, if 0.0.0.0 is routed to itself?

How do you fix this? Create your NAT gateway in the public subnet or make sure the routing table for the subnet the NAT gateway is in (10.0.2.0/24) has the correct route for internet traffic.

You need to have 2 routing tables. One for the subnet the NAT gateway is in. The other routing table is for private subnets that send internet traffic to the NAT gateway.