Ceph radosgw - bucket policy - make all objects public-read by default

Question

I work with a group of non-developers which are uploading objects to an s3 style bucket through radosgw. All uploaded objects need to be publicly available, but they cannot do this programmatically. Is there a way to make the default permission of an object public-read so this does not have to be manually set every time? There has to be a way to do this with boto, but I've yet to find any examples. There's a few floating around using AWS' GUI, but that is not an option for me. :(

I am creating a bucket like this:

#!/usr/bin/env python

import boto
import boto.s3.connection

access_key = "SAMPLE3N84XBEHSAMPLE"
secret_key = "SAMPLEc4F3kfvVqHjMAnsALY8BCQFwTkI3SAMPLE"

conn = boto.connect_s3(
        aws_access_key_id = access_key,
        aws_secret_access_key = secret_key,
        host = '10.1.1.10',
        is_secure=False,
        calling_format = boto.s3.connection.OrdinaryCallingFormat(),
        )

bucket = conn.create_bucket('public-bucket', policy='public-read')

I am setting the policy to public-read which seems to allow people to browse the bucket as a directory, but the objects within the bucket do not inherit this permission.

>>> print bucket.get_acl()
<Policy: http://acs.amazonaws.com/groups/global/AllUsers = READ, S3 Newbie (owner) = FULL_CONTROL>

To clarify, I do know I can resolve this on a per-object basis like this:

key = bucket.new_key('thefile.tgz')
key.set_contents_from_filename('/home/s3newbie/thefile.tgz')
key.set_canned_acl('public-read')

But my end users are not capable of doing this, so I need a way to make this the default permission of an uploaded file.

Answer 1

I found a solution to my problem.

First, many thanks to joshbean who posted this: https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/python/example_code/s3/s3-python-example-put-bucket-policy.py

I noticed he was using the boto3 library, so I started using it for my connection.

import boto3
import json

access_key = "SAMPLE3N84XBEHSAMPLE"
secret_key = "SAMPLEc4F3kfvVqHjMAnsALY8BCQFwTkI3SAMPLE"

conn = boto3.client('s3', 'us-east-1',
                    endpoint_url="http://mycephinstance.net",
                    aws_access_key_id = access_key,
                    aws_secret_access_key = secret_key)

bucket = "public-bucket"

bucket_policy = {
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"AddPerm",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::{0}/*".format(bucket)]
    }
  ]
}

bucket_policy = json.dumps(bucket_policy)
conn.put_bucket_policy(Bucket=bucket_name, Policy=bucket_policy)

Now when an object is uploaded in public-bucket, it can be anonymously downloaded without explicitly setting the key permission to public-read or generating a download URL.

If you're doing this, be REALLY REALLY certain that it's ok for ANYONE to download this stuff. Especially if your radosgw service is publicly accessible on the internet.