Google App Engine and Amazon S3 File Uploads

Question

I know this has been asked before but there is really not a clear answer. My problem is I built a file upload script for GAE and only found out after, that you can only store files up to aprox. 1MB in the data store. I can stop you right here if you can tell me that if I enable billing the 1MB limit is history but I doubt it.

I need to be able to upload up to 20mb per file so I thought maybe I can use Amazon's S3. Any ideas on how to accomplish this?

I was told to use a combination of GAE + Ec2 and S3 but I have no idea how this would work.

Thanks, Max

Answer 1

From the Amazon S3 documentation:

1. The user opens a web browser and accesses your web page.

2. Your web page contains an HTTP form that contains all the information necessary for the user to upload content to Amazon S3.

3. The user uploads content directly to Amazon S3.

GAE prepares and serves the web page, a speedy operation. You user uploads to S3, a lengthy operation, but that is between your user's browser and Amazon; GAE is not involved.

Part of the S3 protocol is a success_action_redirect, that lets you tell S3 where to aim the browser in the event of a successful upload. That redirect can be to GAE.