Bucket name seems to still be in use after deleting it

Question

Using the Java S3 SDK, I can create buckets and put objects in them and list contents and delete buckets. The only thing I cannot do is create the same bucket again after deleting it.

In the stack trace shown below, I have previously created a bucket named gormanm-0709-r-o-o-t, used it, then deleted it via s3.deleteBucket("gormanm-0709-r-o-o-t"). I then attempt to recreated it via s3.createBucket("gormanm-0709-r-o-o-t", "us-standard") but get the error shown below.

com.amazonaws.services.s3.model.AmazonS3Exception: Container gormanm-0709-r-o-o-t exists (Service: Amazon S3; Status Code: 409; Error Code: BucketAlreadyExists; Request ID: 1be8b569-3db4-4eff-bf1e-c4b8dac20272), S3 Extended Request ID: null
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4187)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4134)
	at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:1021)
	at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:973)
	at com.ibm.saas.file.cos.CosFile.createBucket(CosFile.java:794)
	at com.ibm.saas.file.cos.CosFile.ensureBucketExists(CosFile.java:781)
	at com.ibm.saas.file.cos.CosFile.mkdirs(CosFile.java:665)
	at com.ibm.saas.file.cos.CosFileServiceTest.testDir(CosFileServiceTest.java:148)
	at com.ibm.saas.file.cos.CosFileServiceTest.testRootOperations(CosFileServiceTest.java:82)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

I know for sure I deleted it (as the code did not throw any exception and I don't see the bucket appearing in the Bluemix/Softlayer console) and I know for sure no one else "jumped on" my bucket name since I have done this testing over and over again.

My guess is that there is some period of time before the bucket name becomes available for use again but I'm only guessing about that and, even if that were true, there should be some documentation somewhere explaining the rules.

Speaking of documentation, the Getting Help page says to go here (https://developer.ibm.com/answers/smartspace/public-cloud-object-storage/) for questions but that page does not exist.

Answer 1

In IBM COS there is a 10-minute period after bucket deletion where the name is reserved. This delay prevents possible caching issues where a command might write to a deleted bucket. The chances of this are small, and we might shrink this window down a bit, but generally speaking it's not a great practice to be deleting and recreating buckets. Is there a particular reason you need to be able to do this in your workflow?

And yes, as mentioned by @jarmod in the top level comments, this behavior is due to the large number distributed logical appliances that control the slicing and reassembly of objects. The bucket deletion is still mostly immediately consistent, as writes and listings will be not be possible upon receiving the 204 response, but this behavior could be better documented.

While Michael's answer refers to AWS S3, the information is more or less accurate for our implementation as well.

Answer 2

From the S3 Developer Guide:

Bucket ownership is not transferable; however, if a bucket is empty, you can delete it. After a bucket is deleted, the name becomes available to reuse, but the name might not be available for you to reuse for various reasons. For example, some other account could create a bucket with that name. Note, too, that it might take some time before the name can be reused. So if you want to use the same bucket name, don't delete the bucket. [emphasis added]

http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

There is your documentation of the expected delay, and a bonus nugget: you really should not be deleting and recreating buckets. Since the bucket namespace is global, only one bucket of a given name can exist at any given time, and it can of course only be in one region. Buckets are only intended to be deleted when you're never going to need them again -- otherwise, empty the bucket of objects, but keep the bucket. Someone might snap the name up, if it's particularly non-obscure. Also:

The high-availability engineering of Amazon S3 is focused on get, put, list, and delete operations. Because bucket operations work against a centralized, global resource space, it is not appropriate to create or delete buckets on the high-availability code path of your application. It is better to create or delete buckets in a separate initialization or setup routine that you run less often.

http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

Although the details are not public, all S3 regions are in some sense aware of all buckets. If you send an S3 request to the wrong region, that wrong region does know the correct region and will return an error that includes an (apparently undocumented) x-amz-bucket-region: response header.

So when buckets are created -- regardless of region -- there is necessarily some internal communication with the S3 global database that occurs when creating a bucket. A disruption here could presumably prevent the creation of a bucket in any region, even if that region was otherwise operating normally.

Note that -- as of this writing -- another section of the documentation, the S3 Getting Started Guide, provides essentially the same information; however, until at least late 2015, the information provided there was different.

When you delete a bucket, there may be a delay of up to one hour before the bucket name is available for reuse in a new region or by a new bucket owner. If you re-create the bucket in the same region or with the same bucket owner, there is no delay.

https://web.archive.org/web/20150905132716/http://docs.aws.amazon.com/AmazonS3/latest/gsg/DeletingAnObjectandBucket.html

Intuitively, I suspect that the continued growth of S3 could have made this information obsolete prior to its removal, and prior this snapshot from 2015.