How to use spot instance with amazon elastic beanstalk?

Question

I have one infra that use amazon elastic beanstalk to deploy my application. I need to scale my app adding some spot instances that EB do not support.

So I create a second autoscaling from a launch configuration with spot instances. The autoscaling use the same load balancer created by beanstalk.

To up instances with the last version of my app, I copy the user data from the original launch configuration (created with beanstalk) to the launch configuration with spot instances (created by me).

This work fine, but:

1. how to update spot instances that have come up from the second autoscaling when the beanstalk update instances managed by him with a new version of the app?

2. is there another way so easy as, and elegant, to use spot instances and enjoy the benefits of beanstalk?

UPDATE

Elastic Beanstalk add support to spot instance since 2019... see: https://docs.aws.amazon.com/elasticbeanstalk/latest/relnotes/release-2019-11-25-spot.html

Answer 1

I was asking this myself and found a builtin solution in elastic beanstalk. It was described here as follows:

• Add a file under the .ebextensions folder, for our setup we’ve named the file as spot_instance.config (the .config extension is important), paste the content available below in the file https://gist.github.com/rahulmamgain/93f2ad23c9934a5da5bc878f49c91d64
• The value for EC2_SPOT_PRICE, can be set through the elastic beanstalk environment configuration. To disable the usage of spot instances, just delete the variable from the environment settings.
• If the environment already exists and the above settings are updates, the older auto scaling group will be destroyed and a new one is created.
• The environment then submits a request for spot instances which can be seen under Spot Instances tab on the EC2 dashboard.
• Once the request is fulfilled the instance will be added to the new cluster and auto scaling group.
• You can use Spot Advisor tool to ascertain the best price for the instances in use.
• A price point of 30% of the original price seems like a decent level.

I personally would just use the on-demand price for the given instance type given this price is the upper boundary of what you would be willing to pay. This reduces the likelihood of being out-priced and thus the termination of your instances.

This might be not the best approach for production systems as it is not possible to split between a number of on-demand instances and an additional number of spot instances and there might be a small chance that there are no spot instances available as someone else is buying the whole market with high bids.

For production use cases I would look into https://github.com/AutoSpotting/AutoSpotting, which actively manages all your auto-scaling groups and tries to meet the balance between the lowest prices and a configurable number or percentage of on-demand instances.

Answer 2

As of 25th November 2019, AWS natively supports using Spot Instances with Beanstalk.

Spot instances can be enabled in the console by going to the desired Elastic Beanstalk environment, then selecting Configuration > Capacity and changing the Fleet composition to "Spot instance enabled".

There you can also set options such as the On-Demand vs Spot percentage and the instance types to use.

More information can be found in the Beanstalk Auto Scaling Group support page