What's the most scalable way to handle somewhat large file uploads in a Python webapp?

Question

We have a web application that takes file uploads for some parts. The file uploads aren't terribly big (mostly word documents and such), but they're much larger than your typical web request and they tend to tie up our threaded servers (zope 2 servers running behind an Apache proxy).

I'm mostly in the brainstorming phase right now and trying to figure out a general technique to use. Some ideas I have are:

• Using a python asynchronous server like tornado or diesel or gunicorn.
• Writing something in twisted to handle it.
• Just using nginx to handle the actual file uploads.

It's surprisingly difficult to find information on which approach I should be taking. I'm sure there are plenty of details that would be needed to make an actual decision, but I'm more worried about figuring out how to make this decision than anything else. Can anyone give me some advice about how to proceed with this?

Answer 1

If you're open to adding Django to your environment, its uploading file feature has built-in support for uploading files in multiple chunks.

Take a look at the File Uploads overview section of the django documentation.