Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to fix a drifted AWS CloudFormation stack?

How to fix a drifted AWS CloudFormation stack? I modified a BackendECS Service and it is now "drifted" but there's no information on how to resolve this? This is driving me insane? Zero information on how to resolve this?

like image 940
ByteMe Avatar asked Jan 27 '19 07:01

ByteMe


People also ask

What causes CloudFormation drift?

Configuration drift occurs when a manual change is made to your resources outside the CloudFormation stack that created them. As shown in the diagram below, a change to a Lambda function would cause the CloudFormation Stack to drift from the original state.

What is AWS stack drift?

Stack drift occurs when the actual configuration of an infrastructure resource differs from its expected configuration. Typically, this is caused by users editing resources directly by using the underlying service that created the resource.

How do I rerun a failed CloudFormation stack?

In the console, select the stack set that contains the stack on which the operation failed. In the Actions menu, choose Edit StackSet details to retry creating or updating stacks. On the Specify template page, to use the same AWS CloudFormation template, keep the default option, Use current template.

What happens when CloudFormation stack fails?

If stack creation fails, go to the CloudFormation Resources list in the AWS Management Console to find the log group. Note that if stack creation fails before any instances are launched, a log group might not be created. By default, AWS deletes CloudWatch log groups if stack creation fails.


2 Answers

Amazon AWS allows troubleshooting of a drifted stack via its "Drift Detection" feature.

Drift detection can be accessed by navigating to CloudFormation > Select Stack > Actions > Detect Drift for current stack

More on this here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/detect-drift-stack.html

Using this diagnostic tool you can view exactly what is different in your AWS configuration as compared to the state that cloudformation expects.

There are a few ways to potentially resolve this.

1) If you did an automated update via an Infrastructure as code service, rollback your change

2) If you manually updated the ECS Service, change the settings back to the expected state as shown in the drift detection. Once your system is back to the expected state your stack will behave normally

3) Delete the current stack resource and re-create it - this is a dangerous way to resolve this as you will lose your update history and rollback states.

like image 135
sirlanceoflompoc Avatar answered Nov 15 '22 03:11

sirlanceoflompoc


I had a similar issue with cloudformation and likewise there is still a lack of documentation for how to "undrift" the instances back to what the template specified. I found that commenting out the drifted entity allowed cloudformation to "delete" it and then uncommenting it restores it to the desired state.

That said, I'll second the motion to just use terraform instead as enforcing the template is the default behavior whenever you do an apply.

like image 24
Kevin Myers Avatar answered Nov 15 '22 05:11

Kevin Myers