ftp access from AWS VPC private subnet behind a NAT instance

Question

I have created a VPC with public and private subnets on AWS. All app servers are in private subnets and all outbound requests have to be through an internet-facing NAT instance.

At the moment, our project requires the app servers to access a ftp server provided by a service provider.

I have tried several ways to manage that, but all no luck. What I have done was to open a port range, let's say (40000 - 60000) on both NAT and APP security groups, also standard ftp ports 20 - 21 as well.

The user authentication can be passed, but I could not list contents from app servers.

I am able to access the ftp server from NAT, not problem at all.

So what should I do to make it work?

Answer 1

@JohnRotenstein is absolutely correct that you should use Passive FTP if you can. If, like me, you're stuck with a client who insists that you use Active FTP because their FTP site that they want you to connect to has been running since 1990 and changing it now is completely unreasonable, then read on.

AWS's NAT servers don't support a machine in a private subnet connecting using Active FTP. Full stop. If you ask me, it's a bug, but if you ask AWS support they say it's an unsupported feature.

The solution we finally came up with (and it works) is to:

• Add an Elastic Network Interface (ENI) in a public subnet on to your EC2 instance in the private subnet
  • So now your EC2 instance has 2 network adapters, 2 internal IPs, etc.
  • Let's call this new ENI your "public ENI"
• Attach a dedicated elastic IP to your new public ENI
  • Let's assume you get 54.54.54.54 and the new public ENI's internal IP address is 10.1.1.10

• Add a route in your operating system's networking configuration to only use the new public ENI

  • In windows, the command will look like this, assuming the evil active ftp server you're trying to connect to is at 8.1.1.1:

    route add 8.1.1.1 mask 255.255.255.254 10.1.1.1 metric 2
    

  • This adds a route for all traffic to the FTP server at 8.1.1.1 using subnet mask 255.255.255.254 (ie. this IP and only this IP) should go to the internet gateway 10.1.1.1 using ethernet adapter 2 (your second NIC)

• Fed up yet? Yeah, me too, but now comes the hard part. The OS doesn't know it's public IP address for the public EIN. So you need to teach your FTP client to send the PORT command with the public IP. For example if using CURL, use the --ftp-port command like so:

  curl -v --ftp-port 54.54.54.54 ftp://8.1.1.1 --user myusername:mypass
  

And voila! You can now connect to a nightmare active FTP site from an EC2 machine that is (almost entirely) in a private subnet.

Answer 2

Try using Passive (PASV) mode on FTP.

From Slacksite: Active FTP vs. Passive FTP, a Definitive Explanation:

In active mode FTP the client connects from a random unprivileged port (N > 1023) to the FTP server's command port, port 21. Then, the client starts listening to port N+1 and sends the FTP command PORT N+1 to the FTP server. The server will then connect back to the client's specified data port from its local data port, which is port 20.

Thus, the traffic is trying to communicate on an additional port that is not passed through the NAT. Passive mode, instead, creates an outbound connection, which will then be permitted through the NAT