What's best practice for communication between Amazon EC2 instances? [closed]

Question

I've been setting up Amazon EC2 instances for an upcoming project. They are all micro instances, running Ubuntu Server 64bit. Here's what I've setup so far:

• Web Server -- Apache
• Database Server -- MySQL
• Development Server -- Apache & MySQL
• File Server -- SVN & Bacula (backups are done to S3 buckets)

Currently, there's only one Web Server, but eventually there will be more.

My first question is, what is the best, most secure way for Amazon EC2 instances to communicate between each other? Currently I'm using SSH, is that the best method?

According to Amazon, instances communicating between themselves using their Elastic IP addresses will be charged data transfer fees. However, instances communicating using their Private IP addresses can do so for free. Unfortunately, it appears Private IPs change if the instance is stopped and re-started.

So that's my second question, how do you make use of Amazon instances' Private IPs if they're not static?

I know that the instances probably won't be stopped and started very frequently, but still, if the IP address is in various config files, it would be a pain to have to go through them all and change it.

I'm primarily concerned about the Web servers, which will need access to the Database server and the File server, which will need access to all the instances when performing backups.

Note: I've never used Bacula before and I don't have it setup yet, but I'm assuming it will need the IP addresses of the clients to back them up.

Would ServerFault be a better place to post this question?

Answer 1

It's probably more a ServerFault topic, but I'll answer it anyway.

The simplest way to overcome the dynamism of the internal IP addresses is to run a private DNS server in your cluster. My estate uses Windows VMs, so we have an AD pair set up to provide DNS services (as well as, incidentally, centralised VM logon authentication). With each machine being a member of the domain, and with it's DNS pointers set to each of the AD units (primary and backup), the dynamic internal IP is automatically registered with the DNS service whenever it changes - thus, rather than referring to other VMs by their public IP (e.g. 79.125.x.x) which incurs data transfer cost, or by internal IP (e.g. 10.44.x.x) which can change on restart, they use a DNS name (e.g. mydomain.dbserver01) which is persistent.

We use encrypted comms both for internal and external data exchanges - remember, Cloud VMs are by definition inherently less secure than anything inside your corporate firewall, so you should invest more in their security configuration. Our policy is that a Cloud VM should be configured such that it is more secure than an equivalent VM inside our intranet.