Django+gunicorn+nginx upload large file 502 error

Question

Problem

Uploading 1-2mb files works fine. When I attempt to upload 16mb file, i get 502 error after several seconds

More detalied:

1. I click "Upload"
2. Google Chrome uploads file (upload status is changing from 0% to 100% in left bottom corner)
3. Status changes to "Waiting for HOST", where HOST is my site hostname
4. After a half of minute server returns "502 Bad Gateway"

My view:

def upload(request):
    if request.method == 'POST':
        f = File(data=request.FILES['file'])
        f.save()
        return redirect(reverse(display),  f.id)
    else:
        return render('filehosting_upload.html', request)

render(template, request [,data]) is my own shorthand that deals with some ajax stuff;

The filehosting_upload.html:

{% extends "base.html" %}
{% block content %}
    <h2>File upload</h2>
    <form action="{% url nexus.filehosting.views.upload %}" method="post" enctype="multipart/form-data">
        {% csrf_token %}
        <input type="file" name="file">
        <button type="submit" class="btn">Upload</button>
    </form>
{% endblock %}

Logs & specs

There are nothing informative in logs i can find.

Versions:

• Django==1.4.2
• Nginx==1.2.1
• gunicorn==0.17.2

Command line parameters

command=/var/www/ernado/data/envs/PROJECT_NAME/bin/gunicorn -b localhost:8801 -w 4 PROJECT_NAME:application

Nginx configuration for related location:

   location /files/upload {
    client_max_body_size 100m;
    proxy_pass http://HOST;
    proxy_connect_timeout 300s;
    proxy_read_timeout 300s;
   }

Nginx log entry (changed MY_IP and HOST)

2013/03/23 19:31:06 [error] 12701#0: *88 upstream prematurely closed connection while reading response header from upstream, client: MY_IP, server: HOST, request: "POST /files/upload HTTP/1.1", upstream: "http://127.0.0.1:8801/files/upload", host: "HOST", referrer: "http://HOST/files/upload"

Django log

2013-03-23 19:31:06 [12634] [CRITICAL] WORKER TIMEOUT (pid:12829)
2013-03-23 19:31:06 [12634] [CRITICAL] WORKER TIMEOUT (pid:12829)
2013-03-23 19:31:06 [13854] [INFO] Booting worker with pid: 13854

Question(s)

1. how to fix that?
2. is it possible to fix that without nginx upload module?

Update 1 Tried suggested config

 gunicorn --workers=3 --worker-class=tornado  --timeout=90 --graceful-timeout=10 --log-level=DEBUG --bind localhost:8801 --debug

Works fine for me now.

Answer 1

I run my gunicorn with that parameters, try :

python manage.py run_gunicorn --workers=3 --worker-class=tornado  --timeout=90 --graceful-timeout=10 --log-level=DEBUG --bind 127.0.0.1:8151 --debug

or if you run differently, you may run with that options

Answer 2

For large files handling you should use a worker-class. Also I had some trouble using gevent in python 3.7, better to use 3.6.

Django, Python 3.6 example:

Install:

pip install gevent

Run

gunicorn --chdir myApp myApp.wsgi --workers 4 --worker-class=gevent --bind 0.0.0.0:80 --timeout=90 --graceful-timeout=10