How to set S3 bucket policy to (mostly) private when object acl is public?

Question

I can't work out how to set my bucket policy to achieve what I want. Any help would be much appreciated! My desired rules are:

• users in my account have access via user policies, so shouldn't need access specifically granted to them
• anonymous users (or anyone outside my AWS account) should have no access, except:
• one folder /temp_public should have a public GetObject (i.e. if you know the URL you can get the file)
• these policies should override the object ACL on the files in the bucket, as the object ACLs are sometimes set to public read.

The reason for creating the bucket policy is that many of the objects in the bucket have a public read ACL (inadvertently set when the files were uploaded, but could also happen in future so I want to override the object ACL with the bucket ACL).

Ignoring the temp_public folder, I hoped I could just do this:

{
    "Version": "2008-10-17",
    "Id": "Policy123456789",
    "Statement": [
        {
            "Sid": "Stmt1",
            "Effect": "Deny",
            "NotPrincipal": {
                "AWS": "arn:aws:iam::123456789012:root"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::my-bucket-name/*"
        }
    ]
}

(where 123456789012 is my AWS account number), but I get access denied for all users with that bucket policy. I guess the NotPrincipal isn't working in this case?

thanks for any suggestions!

Rory

UPDATE: cross-posted here on AWS forums, and answered!

Answer 1

Many thanks to IP from AWS Forums for this answer, which I've confirmed is working for me:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::BucketName/temp_public/*"
        }
    ]
}

This statement will give anyone Read access to objects inside temp_public folder, regardless ACLs on those files. To override the public access on all other files, you should provide a +Deny+-type statement. The explicit Deny overrides any Allow access, so you must exclude already given permissions. so use NotResource as an exclusion mask (NOT FINAL YET, read below):

{
    "Effect": "Deny",
    "Principal": "*",
    "Action": "s3:GetObject",
    "NotResource": "arn:aws:s3:::BucketName/temp_public/*"
}

However, this will deny access to ALL users including your account too, because principal is set to "*". Therefore, you must exclude your account from this Deny (STILL NOT FINAL):

{
    "Effect": "Deny",
    "NotPrincipal": { "AWS": "arn:aws:iam::XXXXYYYYZZZZ:root" },
    "Action": "s3:GetObject",
    "NotResource": "arn:aws:s3:::BucketName/temp_public/*"
}

(where XXXXYYYYZZZZ is your 12-digit AWS account Id)

There's still problem: the statement above denies access to all IAM users (except root account). You'd like to exclude all your IAM users too, but this is tricky. For some reasons, Amazon S3 doens't support wildcards for specifying IAM users in a bucket policy. You cannot write "arn:aws:iam::XXXXYYYYZZZZ:user/*" as Principal (it gives an error: "Invalid principal in policy"). You have to specify exact user names:

{
    "Effect": "Deny",
    "NotPrincipal": {
        "AWS":  [
                            "arn:aws:iam::XXXXYYYYZZZZ:root",
                            "arn:aws:iam::XXXXYYYYZZZZ:user/user1",
                            "arn:aws:iam::XXXXYYYYZZZZ:user/user2",
                            "arn:aws:iam::XXXXYYYYZZZZ:user/user3", 
                            "arn:aws:iam::XXXXYYYYZZZZ:user/user4" ]
    }
    "Action": "s3:GetObject",
    "NotResource": "arn:aws:s3:::BucketName/temp_public/*"
}

NB from Rory: The S3 docs suggest you can use arn:aws:iam::XXXXYYYYZZZZ:root to cover all users in the account, but that just doesn't seem to work

So the final policy will look like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::BucketName/temp_public/*"
        },
        {
            "Effect": "Deny",
            "NotPrincipal": {
                "AWS":  [
                                    "arn:aws:iam::XXXXYYYYZZZZ:root",
                                    "arn:aws:iam::XXXXYYYYZZZZ:user/user1",
                                    "arn:aws:iam::XXXXYYYYZZZZ:user/user2",
                                    "arn:aws:iam::XXXXYYYYZZZZ:user/user3", 
                                    "arn:aws:iam::XXXXYYYYZZZZ:user/user4" ]
            }
            "Action": "s3:GetObject",
            "NotResource": "arn:aws:s3:::BucketName/temp_public/*"
        }
    ]
}