Using ec2-init scripts with Ubuntu on EC2 - Automatically set hostname and register with Route53

Question

I'd really like to be able to use the ec2-init scripts to do some housekeeping when I spin up an instance. Ideally I'd like to be able to pass user data to set the hostname and run a couple of initialization scripts (to configure puppet etc.).

I see a script called ec2-set-hostname but I'm not sure if you can use it to set an arbitrary hostname from user-data or what the format of the user-data would need to be.

Anyone used these scripts and know how if can set the hostname and run some scripts at the same time?

Thanks in advance.

Answer 1

In the end I decided to skip the ubuntu ec2 scripts and do something similar. I looked into using Amazon's Route53 service as the nameservice and it was really easy to get it up and running.

Using Route53

Here is what I did; Firstly I used the IAM tools to create a user 'route53' with liberal policy permissions for interacting with the Route53 service

Create the dns group & user

iam-groupcreate -g route53 -v
iam-usercreate -u route53 -g route53

Create keys for the user and note these for later

iam-useraddkey -u route53

Give access to the group to add zones and dns records

iam-grouplistpolicies -g route53
iam-groupaddpolicy -p hostedzone -e Allow -g route53 -a route53:* -r '*'

listing the users and policies for a group

iam-grouplistusers -g route53
iam-grouplistpolicies -g route53
iam-grouplistpolicies -g route53 -p hostedzone

To add and remove dns record entries I uses the excellent python wrapper library for Route53, cli53. This takes a lot of the pain out of using route53. You can grab it from here

https://github.com/barnybug/cli53

In my case the python script is symlinked in /usr/bin as cli53. You'll need to set the following environment variables containing keys created earlier for the route53 user.

export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXX

You need to then create a zone entry for your domain e.g. simple.org

cli53.py create simple.org

This should return you an amazon nameserver address that you can associate with your domain name via your domain name registrar, so that hostname lookups for the domain will be redirected to the Route53 servers.

Once the zone is setup, adding and removing entries to it is simple e.g.

cli53 rrcreate simple.org hostname CNAME ec2-184-73-137-40.compute-1.amazonaws.com
cli53 rrdelete simple.org hostname

We use a CNAME entry with the Public DNS name of the ec2 instance as this hostname will resolve to the public IP externally and the private IP from within EC2. The following adds an entry for a host 'test2.simple.org'.

cli53 rrcreate simple.org test2 CNAME ec2-184-73-137-40.compute-1.amazonaws.com --ttl 60 --replace

Automatically set hostname and update Route53

Now what remains is to setup a script to automatically do this when the machine boots. This solution and the following script owes huge debt to Marius Ducea's excellent tutorial found here

http://www.ducea.com/2009/06/01/howto-update-dns-hostnames-automatically-for-your-amazon-ec2-instances/

It's basically doing the same as Marius' setup, but using Route53 instead of Bind.

The script uses the simple REST based services available to each EC2 Instance at

   http://169.254.169.254 

to retrieve the actual Public DNS name and grab the desired hostname from the instance. The hostname is passed to the instance using the customizable 'user-data' which we can specify when we start the instance. The script expects user-data in the format

hostname=test2

The script will

• grab hostname info from the instance user-data
• grab the public DNS name from the instance metadata
• parse out the hostname
• set the hostname to the fully qualified name e.g. test2.simple.org,
• Add a CNAME record for this FQDN in Route53 point to the Public DNS Name
• write an entry into the Messages of the day so that users can see the domain to ec2 mapping when they log in

Copy and save the following as /usr/bin/autohostname.sh

#!/bin/bash

DOMAIN=simple.org

USER_DATA=`/usr/bin/curl -s http://169.254.169.254/latest/user-data`
EC2_PUBLIC=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/public-hostname`
HOSTNAME=`echo $USER_DATA| cut -d = -f 2`

#set also the hostname to the running instance
FQDN=$HOSTNAME.$DOMAIN
hostname $FQDN

export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
export AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxxxxxxxx

# Update Route53 with a CNAME record pointing the hostname to the EC2 public DNS name
# in this way it will resolve correctly to the private ip internally to ec2 and
# the public ip externally
RESULT=`/root/dns/cli53/cli53.py rrcreate $DOMAIN $HOSTNAME CNAME $EC2_PUBLIC --ttl 60 --replace`
logger "Created Route53 record with the result $RESULT"

# write an MOTD file so that the hostname is displayed on login
MESSAGE="Instance has been registered with the Route53 nameservers as '$FQDN' pointing to ec2 domain name '$EC2_PUBLIC'"

logger $MESSAGE

cat<<EOF > /etc/update-motd.d/40-autohostname
#!/bin/bash
# auto generated on boot by /root/bin/auto_hostname.sh via rc.local
echo "$MESSAGE"

EOF

chmod +x /etc/update-motd.d/40-autohostname

exit 0

To get the script to run at boot time, we add a line in /etc/rc.local e.g.

/usr/bin/autohostname.sh

Change the user-data for the test instance to 'hostname=test2' and reboot the instance. Once it reboots, you should be able to login to it via test2.simple.org. It may take a couple of minutes for this to resolve correctly, depending on the TTLs you specified. When you login, you should see a MOTD message telling you

Instance has been registered with the Route53 nameservers as 'test2.simple.org' pointing to ec2 domain name 'ec2-184-73-137-40.compute-1.amazonaws.com'

Once you have this working with the test instance it would make sense to back it up as an AMI that you can use to create other instances with the same autohostnaming abilities.

HTH

Answer 2

I installed the route53 gem, and wrote a little script:

gem install route53

#!/bin/bash
DATE=`date +%Y%m%d%H%M%S`
export HOME=/root
export DEBIAN_FRONTEND=noninteractive
export PATH=/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin:/usr/local/aws/bin    /usr/local/node:$PATH
export JAVA_HOME=/usr/java/current
export EC2_HOME=/usr/local/aws
export EC2_PRIVATE_KEY=/root/.ec2/pk-XXXXXXXXXXXXXXXXXXXXXXX
export EC2_CERT=/root/.ec2/cert-XXXXXXXXXXXXXXXXXXXX
export EC2_INSTANCE_ID=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`
echo "$EC2_INSTANCE_ID"
mkdir  /root/$EC2_INSTANCE_ID
ec2din $EC2_INSTANCE_ID > /root/$EC2_INSTANCE_ID/$EC2_INSTANCE_ID.txt
export FQDN=`cat /root/$EC2_INSTANCE_ID/$EC2_INSTANCE_ID.txt |grep Name |awk '{print $5}'`
export EC2_DNS_NAME=`cat /root/$EC2_INSTANCE_ID/$EC2_INSTANCE_ID.txt |grep INSTANCE |awk '{print $4}'`
/usr/bin/ruby1.8 /usr/bin/route53 -g -z /hostedzone/XXXXXXXX --name $FQDN. --type CNAME --ttl 60 --values $EC2_DNS_NAME > /tmp/route53.out 2>&1

-Josh