Amazon Prefix-Based S3 Policy Isn't Working (AWS, IAM, STS, Ruby)

Question

I'm building an app that uses Amazon's Security Token Service to create temporary users to access a subdirectory on an S3 bucket. The users are created by an IAM user that has full read/write access to the bucket (as well as the permissions necessary to create users).

I have the creation of users working perfectly along with session expiration and more, but I'm having problems getting the right policy in place to allow for prefix-based listing of keys. The permissions that I want the end user to have are:

1. Read objects that are in some defined prefix
2. Write objects to the same defined prefix
3. List all objects that reside in the defined prefix

I managed to get read and write working but somehow no matter what I try the list access isn't working properly. Here's the Ruby code that I was using when I was closest:

AWS::STS::Policy.new do |policy|
  policy.allow(
    actions: ["s3:GetObject*", "s3:PutObject*", "s3:DeleteObject*"],
    resources: "arn:aws:s3:::#{ENV['PROJECT_BUCKET']}/#{folder_path}/*"
  )

  policy.allow(
    actions: ["s3:*"],
    resources: ["arn:aws:s3:::#{ENV['PROJECT_BUCKET']}/*", "arn:aws:s3:::#{ENV['PROJECT_BUCKET']}"]
  ).where(:s3_prefix).like("#{folder_path}/*")
end

If I recall, this allowed me to do reading and writing but not listing. Since I'm still in development I've changed the code to this:

AWS::STS::Policy.new do |policy|
  # FIXME: This is way too permissive, but it's not working to be more specific.
  policy.allow(
    actions: ["s3:*"],
    resources: ["arn:aws:s3:::#{ENV['PROJECT_BUCKET']}/*", "arn:aws:s3:::#{ENV['PROJECT_BUCKET']}"]
  )
end

This works 100% fine with the obvious problem that nothing is constrained to a prefix which would allow users to clobber each other's work.

What am I doing wrong in my policy?

Answer 1

To expand on Bob Kinney's referenced article and fragments (+1), I'd like to explain what I consider the likely cause of your problem, which is actually not related to using the AWS Security Token Service (STS), but involves a few subtleties frequently encountered with Amazon S3 IAM policies in general:

The Example Policies for Amazon S3 cover various use cases similar or related to yours - specifically your use cases apparently includes Example 2: Allow a group to have a shared folder in Amazon S3 - you have effectively implemented that in the first policy of the first fragment already (modulo GetObjectVersion, DeleteObjectVersion, which are only relevant when using Object Versioning).

What's missing now is ListBucket - please note the following subtleties:

• This belongs to Operations on Buckets, i.e. operations you can perform on Amazon S3 buckets , whereas e.g. GetObject belongs to Operations on Objects, i.e. operations you can perform on Amazon S3 objects (furthermore there are Operations on the Service, currently only ListAllMyBuckets, which is likely not applicable to your use case).
• The prefix parameter Limits the response to keys which begin with the indicated prefix. You can use prefixes to separate a bucket into different sets of keys in a way similar to how a file system uses folders., which implies that a prefix cannot contain a wildcard specification, or rather * is just treated as part of the name, see What characters are allowed in a bucket or object name?.
  • This is an aspect of the folders/directories simulation many users stumble upon initially, because S3 is actually a flat storage architecture comprised of buckets and objects/keys only (see my answer to How to specify an object expiration prefix that doesn't match the directory? for more details on that).

Many use cases like yours therefore require two distinct policy fragments to separately address the object and the bucket related operations, accordingly you'll likely need something like the following:

AWS::STS::Policy.new do |policy|
  policy.allow(
    actions: ["s3:GetObject*", "s3:PutObject*", "s3:DeleteObject*"],
    resources: "arn:aws:s3:::#{ENV['PROJECT_BUCKET']}/#{folder_path}/*"
  )

  policy.allow(
    actions: ["s3:ListBucket"],
    resources: ["arn:aws:s3:::#{ENV['PROJECT_BUCKET']}"]
  ).where(:s3_prefix).like("#{folder_path}/")
end

Answer 2

You might find this article of interest, as it specifically discusses creating a policy to restrict users to a prefix in an S3 Bucket.

Credential Management for Mobile Applications

You'll most likely just need to refer to the second policy.

{ 
        "Statement":
        [
            {
              "Effect":"Allow",
              "Action":["s3:PutObject","s3:GetObject","s3:DeleteObject"],
              "Resource":"arn:aws:s3:::__MY_APPS_BUCKET_NAME__/__USERNAME__/*"
            },
            {
              "Effect":"Allow",
              "Action":"s3:ListBucket",
              "Resource":"arn:aws:s3:::__MY_APPS_BUCKET_NAME__",
              "Condition":{"StringLike":{"s3:prefix":"__USERNAME__/"}}
            },
            {
              "Effect":"Deny",
              "Action":["sts:*", "iam:*", "sdb:*"],
              "Resource":"*"
            }
        ]
}

With the first 2 statements being what you'd be most interested in.

Hope this helps.