amazon s3 The specified key does not exist after get_key success

Question

I am using python and boto

this is my code:

key = bucket.get_key(key_name)
if not key:
    print 'error, key does not exist'
    return
data = key.get_contents_as_string()

sometimes (appears randomly) i get this exception:

S3ResponseError: S3ResponseError: 404 Not Found

NOTE: the file is uploaded by one server and then immediately afterwards another server (located in a different continent) is running the code above.

the traceback:

Traceback (most recent call last): File "/test.py", line 222, in _process_response

data = key.get_contents_as_string() File "/usr/lib/python2.6/site-packages/boto-2.1.1-py2.6.egg/boto/s3/key.py",

line 1201, in get_contents_as_string response_headers=response_headers) File "/usr/lib/python2.6/site-packages/boto-2.1.1-py2.6.egg/boto/s3/key.py",

line 1093, in get_contents_to_file response_headers=response_headers) File "/usr/lib/python2.6/site-packages/boto-2.1.1-py2.6.egg/boto/s3/key.py",

line 996, in get_file override_num_retries=override_num_retries) File "/usr/lib/python2.6/site-packages/boto-2.1.1-py2.6.egg/boto/s3/key.py",

line 211, in open override_num_retries=override_num_retries) File "/usr/lib/python2.6/site-packages/boto-2.1.1-py2.6.egg/boto/s3/key.py",

line 165, in open_read self.resp.reason, body) S3ResponseError: S3ResponseError: 404 Not Found

NoSuchKeyThe specified key does not exist.key_nameidhost_id

so i get the key but then when i try and read from it i get 'not found'. any idea ?

Answer 1

This is expected behaviour, according to Amazon S3 developer guide:

... However, information about the changes might not immediately replicate across Amazon S3 and you might observe the following behaviors: A process writes a new object to Amazon S3 and immediately attempts to read it. Until the change is fully propagated, Amazon S3 might report "key does not exist."