Terraform error - RDS Cluster FinalSnapshotIdentifier is required when a final snapshot is required

Question

I am new to Terraform. I am using Terraform to write AWS scripts. I am getting an error while performing Terraform Destroy. Terraform script is

resource "aws_rds_cluster" "aurora-cluster-ci" {
  cluster_identifier        = "aurora-cluster-ci"
  engine                    = "aurora-mysql"
  availability_zones        = ["us-east-1a", "us-east-1b", "us-east-1c"]
  database_name             = "${var.rds_dbname}"
  master_username           = "${var.rds_username}"
  master_password           = "${var.rds_password}"
  backup_retention_period   = 5
  engine_version            = "5.7.16"
  preferred_backup_window   = "07:00-09:00"
  apply_immediately         = true
  final_snapshot_identifier = "ci-aurora-cluster-backup"
  skip_final_snapshot       = true
}

Terraform Destroy throws an error "aws_rds_cluster.aurora-cluster-ci: RDS Cluster FinalSnapshotIdentifier is required when a final snapshot is required"

I have "final_snapshot_identifier" key in my script.

Answer 1

Solution:

I Encountered the same problem while trying to perform a destroy on an RDS instance (not under AWS Aurora) but the principles are the same.

Below are a few steps I took in order to solve this issue:

1. Change skip_final_snapshot to true and remove final_snapshot_identifier if exists
   (see comments #1 and #2 below) .

2. Remove backup_window (Under AWS Aurora its probably called preferred_backup_window).

3. Change backup_retention_period to 0.

4. Make sure that apply_immediately is set to true (see comment #3 below).

5. Run terraform apply and check the changes to affect (see a tip as comment #4 below).

6. Now you can run terraform destroy and no errors should appear (in my case I add deletion_protection set to true and add to remove it).

---

Comment #1 - Understanding the the purpose of the relevant fields

From Terraform docs:

skip_final_snapshot - (Optional) Determines whether a final DB snapshot is created before the DB instance is deleted. If true is specified, no DBSnapshot is created. If false is specified, a DB snapshot is created before the DB instance is deleted, using the value from final_snapshot_identifier. Default is false.

final_snapshot_identifier - (Optional) The name of your final DB snapshot when this DB instance is deleted. Must be provided if skip_final_snapshot is set to false.

In the code specified in the question skip_final_snapshot was true and final_snapshot_identifier was still specified.

(*) Don't be confused with the snapshot_identifier field.

---

Comment #2 - What is causing this error?

For those who want to understand a little bit what is happening here, in the mentioned open issue there is a nice thread where a contributor named @caiges gave a nice explanation there:

For starters, skip_final_snapshot defaults to False which should also require final_snapshot_identifier to be set but it's not so what happens is the create/update is applied, state updated where skip_final_snapshot is False but final_snapshot_identifier is null.
This causes the destroy operation to fail it's verification stage.

This can be fixed but I don't really have a great story for those who already have prexisting state.
One possibility would be that a delete operation ignores skip_final_shopshot if the identifier is null.
Another might be to default final_snapshot_identifier to something random if skip_final_snapshot is set to or defaulted to False.
I think for data safety reasons, ignoring skip_final_snapshot if final_snapshot_identifier is null is a bad idea and it'd be better to just randomize an identifier.

---

Comment #3 - Making sure our changes take immediate effect:

A note about apply_immediately from Terraform's docs:

Note: using apply_immediately can result in a brief downtime as the server reboots. See the AWS Docs on RDS Maintenance for more information.

---

Comment #4 (Bonus) - Saving ourselves some time:

When you run terraform plan make sure that the ~ (update in-place sign) appears in the relevant fields under Terraform's execution plan - In the example below you can see that 2 changes will be applied:

~ resource "aws_db_instance" "postgresql" {
        address                               = ...
        allocated_storage                     = 100
        allow_major_version_upgrade           = false
        .
        .
      ~ apply_immediately                     = false -> true
        .
        .
      ~ backup_retention_period               = 7 -> 0
        .
        .
        tags                                  = ...
        username                              = ...
        vpc_security_group_ids  =  ...
    }

This might sound trivial, but in cases like this error, it can save a lot of debugging time when you try to understand why certain updates haven't took place.

Answer 2

This is a known bug that is still open as of the current version of the Terraform provider for AWS:

https://github.com/terraform-providers/terraform-provider-aws/issues/2588

In a nutshell, it's ignoring the skip_final_snapshot parameter.

Answer 3

In my case I had to manually edit the .tfstate file and set "skip_final_snapshot" to true. Then it worked.

Answer 4

To delete RDS DB from terraform destroy:-

1. first add skip_final_snapshot = "true" to your aws_provider
2. do terraform-apply

Then you are able to destroy it.

3. terraform destroy