How to properly auto-scale AWS EC2 Instances group in a relatively complex infrastructures?

Question

I'm working to migrate our servers on the Amazon Cloud, reasons are auto-scaling possibilities, costs, services, and more, obviously.

So far, I'm experimenting hard and trying to dive in the full-featured documentations, but with no previous experience I've much questions.

The envisaged infrastructure is the following:

                                  +-----+
                                  | ELB |
                                  +--+--+
                                     |
                +--------------------|--------------------+
                |            Auto-Scaling Group           |
                |--------------------|--------------------|
                |                    |                    |
                |  +---------+       |       +---------+  |
                |  | varnish |<------+------>| varnish |  |
                |  +----+----+               +---------+  |
                |       |                         |       |
                +-----------------------------------------+
                        |                         |
                        |                         |
                        |     +------------+      |
                        +---->|Internal ELB|<-----+
                              +------+-----+
                                     |
                +-----------------------------------------+
                |            Auto-Scaling Group           |
                |-----------------------------------------|
                |  +---------+       |       +---------+  |
                |  | Apache  |<------+------>| Apache  |  |
                |  +----+----+               +----+----+  |
                |       |                         |       |
                +-----------------------------------------+
                        |         +-----+         |
                        +-------->| RDS |<--------+
                                  +-----+

In words, I would have Elastic LoadBalancer which will send traffic to the Varnish instances, which will in turn send the traffic to an internal Elastic LoadBalancer which will send the traffic to the Apache frontends.

For now, I've discovered the AWS tools, like the CloudFormation service which seems able to bootstrap instance given a template, this seems great, but it seems able to bootstrap only.

Having a little experience with Puppet (and given the recommandation of AWS on the subject) I dove in the Puppet Master thing which is a great tool.

My idea, which may not be viable or realistic, is to create a "Puppet Node Stack" using CloudFormation templates, which will configure the instance as required and connect the puppet master to be provisioned.

Once I've a stack ready, I'm wondering how to configure/create Auto-Scaling group for both Varnish and Apache instances.

It seems that CFN has resources to configure the auto-scaling groups & policies, so I guess I could create two different templates for each.

But would the AS feature run trough the CFN service, and then does all the init things (and executes the user-data)?

I also read here and there that Puppet can make use of the EC2 Tags, maybe a generic stack template with corresponding tags (like roles) could do the trick?

Is this architecture realistic and viable? Do you have any feedbacks?

Thanks for your advices.

Answer 1

Auto-scaling creates new nodes based on the launch configuration. So you would have two separate auto scaling groups and two separate launch configurations. ie

"VarnishScalingGroup" : {
  "Type" : "AWS::AutoScaling::AutoScalingGroup",
  "Properties" : {
    "LaunchConfigurationName" : {"Ref" : "VarnishLaunchConfiguration" },
    "LoadBalancerNames" : {"Ref" : "ELB"},
    ...
  }
},
"VarnishLaunchConfiguration" : {
  "Type" : "AWS::AutoScaling::LaunchConfiguration",
  "Properties" : {
    ...
    "UserData" : {
      ....
    },
    "MetaData" : {
      ...
    }
 },
"ApacheScalingGroup" : {
  "Type" : "AWS::AutoScaling::AutoScalingGroup",
  "Properties" : {
    "LaunchConfigurationName" : {"Ref" : "ApacheLaunchConfiguration" },
    "LoadBalancerNames" : {"Ref" : "InternalELB"},
    ...
  }
},
"ApacheLaunchConfiguration" : {
  "Type" : "AWS::AutoScaling::LaunchConfiguration",
  "Properties" : {
    ...
    "UserData" : {
      ....
    },
    "MetaData" : {
      ...
    }
 }

The other thing you'd want to add is separate scaling policies for each scaling group, and appropriate CloudWatch metrics to match.

CloudFormation can also initiate updates to the stack. If as part of the userdata you kick of cfn-hup, then it will periodically (you decide) check for changes in the stack meta data - and then execute whatever you prefer. I tend to kick off another version of cfn-init - which will parse and update any meta data.

Key point - if you go down the cfn-hup path, it will not execute userdata again, unless the CloudFormation stack requires dropping and creating new instances.

One other point, if you want updates to the LaunchConfiguration to be rolled out, you need to ensure that the LaunchConfiguration also has an UpdatePolicy applied to it.