Architectural and design question about uploading photos from iPhone app and S3

Question

I want to allow users of an iPhone app to upload photos and use Amazon S3. There are 2 ways I see going about this:

1. Upload from iPhone to my server, which proxies it then to Amazon S3.
2. Upload from iPhone direct to S3

For option 1, the security is straightforward. I don't ever have to tell the iPhone my S3 secret. Downside is that everything is proxied through our server for uploads which sort of defeats the purpose of going to S3.

For option 2, in theory it's better but the issue is how do you enable the iPhone (or any mobile app on a different platform) to write into my S3 bucket without giving it my secret? Additionally, I'm not sure if this is a good design or not because the flow would be: iphone uploads to S3, gets the URL, then tells the server what the URL is so it can add it to our database to reference in the future. However, since the communication is separated into 2 legs (iphone->S3 vs iPhone->My-Server) it leaves it fragile as a non-atomic operation.

I've found some older info that references using Browser-based Uploads using POST but unsure if that is still the recommended approach. I'm hoping for a better solution where we can just use the REST APIs rather than relying on POST. I've also see the AWS iOS Beta SDK, but their docs didn't help much and I found an Amazon article that was equally unhelpful as it cautions you on what not to do, but doesn't tell you an alternative approach:

The mobile AWS SDKs sign the API requests sent to Amazon Web Services (AWS) in order to validate the identity of the AWS account making the request. Otherwise, a malicious developer could easily make requests to another developer's infrastructure. The requests are signed using an AWS Access Key ID and a Secret Access Key that AWS provides. The Secret Access Key is similar to a password, and it is extremely important to keep secret.

Tip: You can view all your AWS security credentials, including Access Key ID and Secret Access Key, on the AWS web site at http://aws.amazon.com/security-credentials.

Embedding credentials in source code is problematic for software, including mobile applications, because malicious users can de-compile the software or view the source code to retrieve the Secret Access Key.

Does anyone have any advice on the best architectural design and flow for this?

Update: The more I dig into this, it seems that a bunch of pople recommend using the HTTP POST method with the json policy file as described here: http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/index.html?UsingHTTPPOST.html.

With this, the flow would be something like (1) iPhone makes request to my server, asking for policy file (2) server generates json policy file and gives back to client (3) iPhone does HTTP POST of photo + json policy to S3. I hate that I'm using HTTP POST in an apparently kludgy way but it appears to be better because it removes the need for my server to store the photo at all.

Answer 1

I've discussed this issue on the AWS forums before. As I say there, the proper solution for accessing AWS from a mobile device is to use the AWS Identity and Access Management service to generate temporary, limited-privilege access keys for each user. The service is great, but it's still in beta for now and it's not part of the mobile SDK yet. I have a feeling once this thing is released for good, you'll see it out on the mobile SDK immediately afterwards.

Until then, generate presigned URLs for your users, or proxy through your own server like some others have suggested. The presigned URL will allow your users to temporarily GET or PUT to an S3 object in one of your buckets without actually having your credentials (they are hashed into the signature). You can read about the details here.

EDIT: I've implemented a proper solution for this problem, using the preview beta of IAM. It's available on GitHub, and you can read about it here.

Answer 2

Upload to your server and then post to S3. From an architecture standpoint you will want to do this from your server. There are many things that could go wrong during the data transfer you can handle better on the server and if you want to store any data about the image you are sending to S3 you are probably going to have a server side call anyway.

Plus, your Secret Access Key is stored in a more secure environment. Your own.

If you are worried about scalability and you are going to be doing a considerable number of S3 transfers I would consider hosting your server on and EC2 instance. Transferring data between the two is free (given you store you data in following data centers).

There is no Data Transfer charge for data transferred between Amazon EC2 and Amazon S3 within the same Region or for data transferred between the Amazon EC2 Northern Virginia Region and the Amazon S3 US Standard Region." Amazon Simple Storage Service (Amazon S3)

There are many post here on SO Amazon - EC2 cost? (example) about the pros and cons of using EC2.