Why are my Amazon S3 key permissions not sticking?

Question

I'm using the Python library boto to connect to Amazon S3 and create buckets and keys for a static website. My keys and values are dynamically generated, hence why I am doing this programmatically and not through the web interface (it works using the web interface). My code currently looks like this:

import boto
from boto.s3.connection import S3Connection
from boto.s3.key import Key

conn = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
bucket = conn.create_bucket(BUCKET_NAME)
bucket.configure_website('index.html', 'error.html')
bucket.set_acl('public-read')

for template in ['index.html', 'contact-us.html', 'cart.html', 'checkout.html']:
    k = Key(bucket)
    k.key = key
    k.set_acl('public-read')
    k.set_metadata('Content-Type', 'text/html')
    k.set_contents_from_string(get_page_contents(template))

I'm getting various errors and problems with this code. When the keys already existed and I used this code to update them, I would set the ACL of each key to public-read, but I'd still get 403 forbidden errors when viewing the file in a browser.

I tried removing all the keys to recreate them from scratch, and now I get a NoSuchKey exception. Obviously the key isn't there because I'm trying to create it.

Am I going about this the wrong way? Is there a different way of doing this to create the keys as opposed to updating them? And am I experiencing some kind of race condition when the permissions don't stick?

Answer 1

I'm still not completely sure why the code above didn't work, but I found a different (or newer?) syntax for creating keys. The order of operations also appears to have some effect. This is what I came up with that worked:

conn = S3Connection(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
bucket = conn.create_bucket(store.domain_name)
bucket.set_acl('public-read')
bucket.configure_website('index.html', 'error.html')

for template in ['index.html', 'contact-us.html', 'cart.html', 'checkout.html']:
    k = bucket.new_key(template)
    k.set_metadata('Content-Type', 'text/html')
    k.set_contents_from_string(get_page_contents(template))
    k.set_acl('public-read') #doing this last seems to be important for some reason

Answer 2

This has bitten me as well. boto's set_contents_from_string() method apparently sets the key's ACL to private, overriding any existing ACL.

So if you do set_acl('public-read'), followed by a set_contents_from_string(), the 'public-read' will be overridden.