CDK stack stuck in UPDATE_ROLLBACK_FAILED - How to continue?

Question

I have a CDK project. I tried to deploy a stack but it had an error related to permissions. Now the stack state is UPDATE_ROLLBACK_FAILED. I fixed the error, and I would like to continue my deployment. When I enter cdk deploy it fails:

is in UPDATE_ROLLBACK_FAILED state and can not be updated

I read here that you can issue a command to ContinueUpdateRollback. Can I do this in CDK? What's the best practice for such a state? What do I do if this happens in production, I don't want to delete the stack...

Answer 1

You cannot fix this in CDK -- although you can use the AWS cloudformation CLI to run continue-update-rollback, usually this state shouldn't be resolved programmatically because it requires a decision on your part. You have to go into the console > Cloudformation > Your Stack > Stack Actions > Continue Rollback to see what cannot be updated/rolled back.

You will be provided with a prompt as to what the exact issue is (usually a resource that cannot be updated/deleted etc). You can choose to skip updating this resource and the rollback will continue and succeed.

To see what went wrong with the deployment, or what tried to change that you weren't expecting, run

cdk deploy --no-execute --change-set-name debug-changeset

This command will not actually deploy. It will just generate a change set (called debug-changeset) that you can view from Cloudformation console of the stack in question. This file will show you what cdk deploy wanted to change and can help you debug why your update failed (I debug it this way because I find the deployment events log errors are usually not detailed enough to help you figure out the exact problem)