How do I configure my uWsgi server to protect against the Unreadable Post Error?

Question

This is the problem:

  File "/app/.heroku/python/lib/python2.7/site-packages/django/utils/six.py", line 535, in next
    return type(self).__next__(self)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/multipartparser.py", line 344, in __next__
    output = next(self._producer)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/utils/six.py", line 535, in next
    return type(self).__next__(self)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/multipartparser.py", line 406, in __next__
    data = self.flo.read(self.chunk_size)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/request.py", line 267, in read
    six.reraise(UnreadablePostError, UnreadablePostError(*e.args), sys.exc_info()[2])
  File "/app/.heroku/python/lib/python2.7/site-packages/django/http/request.py", line 265, in read
    return self._stream.read(*args, **kwargs)
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 59, in read
    result = self.buffer + self._read_limited(size - len(self.buffer))
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 47, in _read_limited
    result = self.stream.read(size)
UnreadablePostError: error during read(65536) on wsgi.input

My current configuration reads like this:

[uwsgi]
http-socket = :$(PORT)
master = true
processes = 4
die-on-term = true
module = app.wsgi:application
memory-report = true
chunked-input-limit = 25000000
chunked-input-timeout = 300
socket-timeout = 300

Python: 2.7.x | uWsgi: 2.0.10

And to make the problem even more specific, this is happening when I process images synchronously along with an image upload. I know that ideally I must do this using Celery, but because of a business requirement I am not able to do that. So need to configure the timeout in such a way that it allows me to accept a large image file, process it and then return response.

Any kind of light on the question will be extremely helpful. Thank you.

Answer 1

The error quoted in the description isn't the full picture; the relevant part is this lot entry:

[uwsgi-body-read] Error reading 65536 bytes … message: Client closed connection uwsgi_response_write_body_do() TIMEOUT

This specific error is being raised because (most probably) the client, or something between it and uWSGI, aborted the request.

There are a number of possible causes for this:

• A buggy client
• Network-level filtering (DPI or some misconfigured firewall)
• Bugs / misconfiguration in the server in front of uWSGI

The last one is covered in the uWSGI docs:

If you plan to put uWSGI behind a proxy/router be sure it supports chunked input requests (or generally raw HTTP requests).

To verify your issue really isn't in uWSGI, try to upload the file via the console on the server hosting your uWSGI application. Hit the HTTP endpoint directly, bypassing nginx/haproxy and friends.