Is there conflict between bucket policies and signed urls?

Question

Some files in my bucket are set to public-read (ACL). So I read somewhere that setting a bucket policy can automatically set all files in a bucket to private.

bucketname is a placeholder for the actual bucket name. My bucket policy is:

{
    "Version": "2008-10-17",
    "Id": "Policy1331182170360",
    "Statement": [
        {
            "Sid": "Stmt1331182162671",
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::bucketname/*"
        }
    ]
}

In one of my controllers, I have:

    s3 = Aws::S3.new(APP_CONFIG['amazon_access_key_id'], APP_CONFIG['amazon_secret_access_key'])
    bucket_gen = Aws::S3Generator::Bucket.create(s3, APP_CONFIG['amazon_bucket_name'])
    signed_url = bucket_gen.get("#{URI.unescape(URI.parse(URI.escape(@song.encoded_file_url)).path[1..-1])}", 10.minute) 
    redirect_to signed_url and return

I am re-directed, but I keep getting access denied. However, if I remove the bucket policy, I am re-directed to a signed url and everything works fine.

Initially, I thought there was a problem with the way I was signing my urls. So I opened Amazon Web Console, and manually set the file permissions to private. For this test, I removed the bucket policy. Navigated to the file url (not signed) and couldn't access. Which is normal. On the second test, I signed the url and could access the file. Which means theres nothing wrong with the way I was signing.

Is there a conflict between bucket policies and signed urls?

Answer 1

This boils down to authentication (who are you) being different from authorization (what are you allowed to do). A signed URL is merely a way of authenticating yourself as a particular AWS user (as opposed to just being anonymous ), it doesn't confer any special authorization.

Once authentication has taken place and amazon knows who you are, it still needs to decide whether you are allowed to perform the requested action. Your bucket policy says that no-one can access files and so the request is denied see the policy docs for why your bucket policy evaluates that way

If your policy said that your user can access files but that no-one else can then your signed urls should work (which is basically what private means when you do individual file permissions)

If you add a statement to your policy along the lines of

{
  "Sid":"AddSpecific",
  "Effect":"Allow",
  "Principal": {
    "AWS": ["1234-5678-9012"]
   },
  "Action":["Action": "s3:GetObject"],
  "Resource":["arn:aws:s3:::bucket/*"]
}

then you should be able to use your signed urls (obviously replace 1234-5678-9012 with your aws account id. If you are using IAM then the iam bit of web console has the identifiers for those users)