Speed up AMI and ASG Creation

Question

using Ansible I create an AMI of a ubuntu instance then using this AMI to create an Launch configuration and then update and auto scaling group, is there any shortcuts I can take to speed up the ASG and AMI steps, take 10mins+

Answer 1

I asked a similar question via an AWS Support ticket, here was there response:

The main time consuming process when launching up a new EC2 instance is the boot process of the OS within the instance: having more or less security groups/network ACLs, different amount of SSH keypairs, and roles associated to the instance should have no measurable impact in the time it takes for it to start up. Most of the time it takes to launch an instance is consumed by the OS itself.

With that in mind, allow me to go over some of the main items that might consume the most time when an instance boots - for all the points below I will focus exclusively on Linux from an EC2 point of view:

• Local filesystem mounts: if your instance needs to mount a large quantity of filesystems, this will add a little time to your boot process. Depending on the filesystem types you use, you might need to run a check periodically every fixed number of days - for example, on Linux you might need to run fsck on an ext4 filesystem every 90 days (this period can change depending on your settings), and the OS automatically triggers this check when the filesystem is mounted upon boot if it detects that the period has been exceeded. One strategy for preventing this can be executing these checks before you create the AMI you'll use to launch your new instances, so that any instances launched from this AMI will have their filesystems recently checked and you won't run into unexpected fsck executions when launching your instances for the first time. Depending on the type of filesystem you use, it may be possible to disable these periodic checks altogether, but I wouldn't recommend it as they are necessary to maintain the filesystem's integrity over time.

• Remote filesystem mounts: if your instance needs to mount any remote filesystem (for example, an EFS share, or any conventional NFS share), your boot process may be delayed depending on the network connectivity to the server sharing this remote filesystem. In the worst case scenario, if the server sharing the filesystem is offline, your boot process will be interrupted until this connection times out and fails. If you're mounting any remote filesystems by default when launching your instances, make sure the servers sharing them are available before launching your instances.

• Regular OS initialization scripts: the largest part of the time consumed by the boot process will be taken by the starting the OS services. There are two types models for this in Linux: the traditional SystemV init (which starts services in a serial way, one after the other), and the relatively new systemd (which is able to start services in parallel, and thus decrease the boot time in some circumstances). Which of these you use will depend on the Linux distribution you run in your instances. For example, if you need to start a DB server that might take a long time every time you boot your instance, it might be worthwhile to consider systemd as it will allow for the rest of the unrelated services to be launched in parallel, as long as they don't have this DB server as a prerequisite.

• User data and cloud-init scripts: these are executed after the regular OS init scripts are over. Like on the previous items, you might want to check that any of these custom scripts you execute are optimized so they can take the least time necessary; it's recommendable to test any custom user data scripts individually to measure their time before adding them to a new instance launch so you can have a better idea of their impact in the overall time of the instance startup. If your scripts are retrieving any files external to the instance (if you download something from an S3 bucket, run an automatic update of the installed packages, etc), the same considerations I've mentioned of the "Remote filesystem mounts" item mentioned above apply - make sure there are no network issues that could slow down or prevent this connection, as this would add more time to the overall startup process of your instance.

• Instance type: instance type does affect the time it takes for the OS to finish booting. Under the same circumstances, a t2.large instance will boot faster than a t2.nano simply because it has more RAM, vCPUs, and a higher amount of CPU credits - all of which allow the OS to execute the boot process faster. Also, if you need to retrieve large amounts of data as part of the instance launch process, you might want to use enhanced networking mode and EBS-optimized instances to ensure you have the best bandwidth available for your needs; see the links at the end of this message for more details about this.

• EBS Volume type: just like with instance type, the performance of your EBS volumes is also a factor that can impact on the overall time of the instance startup times. If your instance needs to read large amounts of data during the boot process from a sc1 volume (an HDD volume with low performance designed for infrequently accessed data), the boot process will be slower than if your instance reads the same data from a PIOPS volume with a much higher performance - if your use case contains a scenario where you are affected by this, you might want to test different volume types to choose the one that better suits your needs. Likewise, the type of your instance's root volume will also impact on the boot performance, since in all cases you will have to read information from it. For most cases, the default "General purpose SSD" a.k.a. gp2 EBS volumes are good enough for instance root volumes.

Ultimately, the time taken to launch a new instance will be determined by running benchmarks for your particular use case; the general considerations I've mentioned above can help you reduce this time, but to determine which settings that are best for your environment you'll need to test and fine tune each parameter until you reach a point where the startup time suits your needs.

I'm attaching a couple of links at the end of this reply with more details about the items I've mentioned on this reply.

I hope this information has been useful to you. Please let me know if you have any questions.

Thanks,

Related links: - EC2 instance types: https://aws.amazon.com/ec2/instance-types/ - EBS volume types: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html - EBS optimized instances: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html - Performance tips for EBS volumes: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerformance.html - Enhancing networking mode on EC2 Linux instances: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html