How to force delete a non empty S3 bucket with versioning enabled?

Question

I have this code block in my CDK file:

const artifactBucket = new s3.Bucket(this, "MyBucket", {
  bucketName: "cdk-cws-pipeline-artifacts",
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  versioned: true, // a bucket used as a source in CodePipeline must be versioned
});

// code omitted for brevity

const pipeline = new codepipeline.Pipeline(this, "Pipeline", {
  artifactBucket: artifactBucket,
  // code omitted for brevity
});

Everything was working fine until I try to delete the stack. I got this error:

Are you sure you want to delete: cdk-staging-CwsPipeline (y/n)? y
cdk-staging-CwsPipeline: destroying...
6:11:42 PM | DELETE_FAILED        | AWS::S3::Bucket             | MyBucketF68F3FF0
The bucket you tried to delete is not empty. You must delete all versions in the bucket. (Service: Amazon S3; Status Code: 409; Error Code: BucketNotEmpty; R
equest ID: 1P2N8WFPFQ8R9J4W; S3 Extended Request ID: uQWJgUnsgxVAXoEM95L28jm8p0qGbX+F0oNzMt8BH6Bg5G4Vd6mbNenJdDmPIgtrhXJFUbveSpk=)

    new Bucket (/Users/zulh/hg/cws-cdk/node_modules/@aws-cdk/aws-s3/lib/bucket.ts:1049:22)
    \_ new CiCdStack (/Users/zulh/hg/cws-cdk/lib/ci-cd-stack.ts:128:32)
    \_ Object.<anonymous> (/Users/zulh/hg/cws-cdk/bin/cws-cdk.ts:99:24)
    \_ Module._compile (internal/modules/cjs/loader.js:1200:30)
    \_ Module.m._compile (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/index.ts:858:23)
    \_ Module._extensions..js (internal/modules/cjs/loader.js:1220:10)
    \_ Object.require.extensions.<computed> [as .ts] (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/index.ts:861:12)
    \_ Module.load (internal/modules/cjs/loader.js:1049:32)
    \_ Function.Module._load (internal/modules/cjs/loader.js:937:14)
    \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    \_ main (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/bin.ts:227:14)
    \_ Object.<anonymous> (/Users/zulh/hg/cws-cdk/node_modules/ts-node/src/bin.ts:513:3)
    \_ Module._compile (internal/modules/cjs/loader.js:1200:30)
    \_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1220:10)
    \_ Module.load (internal/modules/cjs/loader.js:1049:32)
    \_ Function.Module._load (internal/modules/cjs/loader.js:937:14)
    \_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
    \_ /Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/npm/node_modules/libnpx/index.js:268:14

6:11:42 PM | DELETE_FAILED        | AWS::CloudFormation::Stack  | cdk-staging-CwsPipeline
The following resource(s) failed to delete: [MyBucketF68F3FF0].

 ❌  cdk-staging-CwsPipeline: destroy failed Error: The stack named cdk-staging-CwsPipeline is in a failed state: DELETE_FAILED (The following resource(s) failed to delete: [MyBucketF68F3FF0]. )
    at Object.waitForStack (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/lib/api/util/cloudformation.ts:266:11)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at Object.destroyStack (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:365:28)
    at CdkToolkit.destroy (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:251:9)
    at main (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/bin/cdk.ts:270:16)
    at initCommandLine (/Users/zulh/.nvm/versions/node/v14.4.0/lib/node_modules/aws-cdk/bin/cdk.ts:185:9)
The stack named cdk-staging-CwsPipeline is in a failed state: DELETE_FAILED (The following resource(s) failed to delete: [MyBucketF68F3FF0]. )

Is there any way to force delete the S3 bucket using CDK TypeScript code when running cdk destroy stack-name?

Answer 1

Now, you can delete the bucket as well as the contents of the Bucket which got created with the help of CloudFormation (i.e. cdk deploy)

You just have to add autoDeleteObjects: true parameter while creating the S3 object.

Here is the Sample TypeScript Code:

new s3.Bucket(this, 'MyFirstBucket', {
  versioned: true,
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  autoDeleteObjects: true
});

Here is the reference link from the official AWS Documentation: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-s3-readme.html#bucket-deletion

Answer 2

Unfortunately you can't do this easily. You can't delete a non-empty bucket using CloudFormation or CDK, as you already experienced.

If you want your stack to continue deletion, rather than fail on the bucket, the easiest option is to add RemovalPolicy of RETAIN. This will simply skip the bucket. Stack will get deleted, but bucket will remian. Later you can use CLI or console to delete it.

If you really want to do everything from CDK or CloudFormation you would have to develop a custom solution for that. This can be done using custom-resource. The resource would be based on a lambda function which you would have to develop and deploy. The lambda would be able to empty the bucket when stack is deleted. Example code of how to delete versioned objects in a bucket using boto3 here.

Hope this helps.