How to fix a drifted AWS CloudFormation stack? I modified a BackendECS Service and it is now "drifted" but there's no information on how to resolve this? This is driving me insane? Zero information on how to resolve this?
Configuration drift occurs when a manual change is made to your resources outside the CloudFormation stack that created them. As shown in the diagram below, a change to a Lambda function would cause the CloudFormation Stack to drift from the original state.
Stack drift occurs when the actual configuration of an infrastructure resource differs from its expected configuration. Typically, this is caused by users editing resources directly by using the underlying service that created the resource.
In the console, select the stack set that contains the stack on which the operation failed. In the Actions menu, choose Edit StackSet details to retry creating or updating stacks. On the Specify template page, to use the same AWS CloudFormation template, keep the default option, Use current template.
If stack creation fails, go to the CloudFormation Resources list in the AWS Management Console to find the log group. Note that if stack creation fails before any instances are launched, a log group might not be created. By default, AWS deletes CloudWatch log groups if stack creation fails.
Amazon AWS allows troubleshooting of a drifted stack via its "Drift Detection" feature.
Drift detection can be accessed by navigating to CloudFormation > Select Stack > Actions > Detect Drift for current stack
More on this here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/detect-drift-stack.html
Using this diagnostic tool you can view exactly what is different in your AWS configuration as compared to the state that cloudformation expects.
There are a few ways to potentially resolve this.
1) If you did an automated update via an Infrastructure as code service, rollback your change
2) If you manually updated the ECS Service, change the settings back to the expected state as shown in the drift detection. Once your system is back to the expected state your stack will behave normally
3) Delete the current stack resource and re-create it - this is a dangerous way to resolve this as you will lose your update history and rollback states.
I had a similar issue with cloudformation and likewise there is still a lack of documentation for how to "undrift" the instances back to what the template specified. I found that commenting out the drifted entity allowed cloudformation to "delete" it and then uncommenting it restores it to the desired state.
That said, I'll second the motion to just use terraform instead as enforcing the template is the default behavior whenever you do an apply.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With