Looking for advice on Amazon S3 bucket setup and management in a Rails multi-tenancy app

Question

Each tenant will have their own photo gallery which stores photos on Amazon S3. Seeing as S3 is relatively new to me I'm looking for some advice and best practices on how to manage this in terms of buckets, IAM groups/users, security, usage reporting, and possibly billing.

The way I see it is I have two options.

Option 1: One master bucket. Each tenant has a sub-directory where their photos are stored. I would have one IAM group for the whole application and create a new IAM user for each tenant with access to only their sub-directory. In the future if I want to know how much S3 space a tenant is using will it be easy to report on? Would I want to have a unique AWS access key and secret key for each tenant even though they are going to the same bucket?

Option 2: Each tenant gets their own bucket. Each tenant would get their own IAM user with access only to their bucket. Is this option better for reporting on usage?

General questions:

• Are there any major drawbacks to either option?
• Is there another option I'm unaware of?
• Can I report on storage via an IAM user's activity or does it happen at the bucket level?

Answer 1

I think you're trying to turn your S3 account into a multi-user thing, which it's not.

Each tenant gets their own bucket

You are limited to 100 buckets, so this is probably not what you want. (Unless it's a very exclusive web service :)

One master bucket

OK

IAM user for each tenant

Um, I think there's a limit for IAM users too.

if I want to know how much S3 space a tenant is using will it be easy to report on?

You can write a script easy enough.

billing

You can use DevPay buckets, in which case you can have 100 buckets per user. But this requires each user sign up for AWS and other complications.

Can I report on storage via an IAM user's activity or does it happen at the bucket level?

IAM is only checked at "ingress". After that, it's all just "your account". So the files don't have different "owners".

Is there another option I'm unaware of?

The usual way is to have a thin EC2 service that controls the security:

• You write a web app and run it on EC2. It knows how to authenticate your users.
• When a user wants to upload, they either POST it to EC2 (and it copies to S3, and probably resizes it anyway). Or you generate a signed POST/PUT URL for the browser to directly upload into S3 (really easy to do once you understand.)
• When a user wants to view a file, they hit your service to get a signed URL that allows them access to their file. But that access times out after a while. That's OK, since they are only accessing the files via your EC2 webpage.

The upshot is that your EC2 box can be small because it's just creating URLs for the browser.