How to configure S3 bucket policies for my app

Question

I have a bucket called prod, with a directory tree that looks like this:

prod
 logs
 uploads
  user
   doc
    1
     a.jpg
     b.jpg
    2
     a.jpg
     b.jpg
  thing
   photos
    1
     a.jpg
     b.jpg
    2
     a.jpg
     b.jpg

Everything in thing/photos should be public. GET requests should be allowed for everyone, but PUT and POST requests should only be allowed when users upload a file through my app.

The user/doc directory on the other hand, I want to be completely private. POST requests should be allowed for users who upload a file through my app, but the only person who should be able to GET those resources is an admin. That data is encrypted before it is stored, but I want to make sure that the folder is not accessible to the public or to other users of my app.

After reading A deep dive into AWS S3 access controls – taking full control over your assets and ACLs - What Permissions Can I Grant? I remain confused about how to accomplish what I want. The overlapping access controls leave me feeling bewildered, and I cannot find a tutorial that explains any of this in an action-oriented approach. Given the number of data leaks caused by improperly set S3 bucket policies, it seems likely that I'm not the only person who misunderstands this.

How are your policies set? Do you have a link to a better tutorial than the ones I've found? Thank you!

Answer 1

Amazon S3 buckets are private by default. Therefore, access is only available if you specifically configure access.

Everything in thing/photos should be public

If you wish to make an entire bucket, or part of a bucket public, use a Bucket Policy.

Copying from @avlazarov's answer:

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Sid":"Example",
      "Effect":"Allow",
      "Principal": "*",
      "Action":["s3:GetObject"],
      "Resource":["arn:aws:s3:::examplebucket/things/photos/*"]
      }
  ]
}

This means:

• Allow anyone
• To GetObject, meaning read an object
• As long as it is in the bucket named examplebucket, in the things/photos/ path

Please note that they will not be able List the contents of the path, so they will need to know the exact name of the object they are retrieving.

Before adding a Bucket Policy, you will need to deactivate the setting in Amazon S3 block public access that prevents Bucket Policies being added.

General rule: When granting public access, use a Bucket Policy.

---

The user/doc directory should be completely private

Amazon S3 buckets are private by default. Therefore, nothing needs to be done.

However, you then mention that the mobile app should have access. Such permissions should be granted via Identity and Access Management (IAM) settings.

Since you mention 'users', there is probably some authentication method being used by your app, presumably to a back-end service. Therefore, rather than putting IAM credentials directly in the app, the flow should be:

• User logs into the app
• The app sends the authentication information to a back-end service that authenticates the user (could be Cognito, login with Google or even just your own database)
• If the user is verified, then the back-end service would generate temporary credentials using the AWS Security Token Service (STS). A policy can be attached to these credentials, granting sufficient permissions for the user and app for this particular session. For example, it could grant access to a path (sub-directory) so that the user can only access objects in their own sub-directory.

It is preferable to only grant the app (and therefore the user) the minimal amount of permissions required to use the service. This avoids intentional or accidental problems that might be caused by providing too much access.

General rule: Only provide mobile apps the minimum permissions they require. Assume that accidents or intentional hacking will happen.

---

The only person who should be able to GET those resources is an Admin

When granting access to your own staff, use policies attached to an IAM User or IAM Group.

I would recommend:

• Create an IAM Group for Admins
• Attach an IAM policy to the Group that grants desired access
• Create an IAM User for each of your staff admins
• Put their IAM User in the IAM Group

This way, all admins (including future ones) will obtain appropriate access, and you can track what each IAM User did independently. Never have multiple staff use the same logins. It would also be advisable to associate a Multi-factor Authentication device to each admin account since the permissions could be dangerous if access was compromised. MFA can be as simple as running an authentication app on a phone that provides a number that changes every 30 seconds.

In fact, some companies only give the Admins 'normal' accounts (without superpowers). Then, if they need to do something extraordinary, they have the Admins temporarily switch to an IAM Role that gives 'admin' capabilities. This minimizes the chance of accidentally doing something that might have an impact on the system.

General rule: Use IAM to grant access to staff within your organization.

If you wish to learn more about IAM, I would highly recommend IAM videos from the annual AWS re:Invent conference. For a complete list of sessions, see: AWS re:Invent 2019 Sessions & Podcast Feed