uWSGI raises OSError: write error during large request

Question

My application uses nginx, with uWSGI on the server side. When I do a large request (with a response time > 4s), the following appears:

SIGPIPE: writing to a closed pipe/socket/fd (probably the client
    disconnected) on request _URL_ (ip XX.XX.XX.XX) !!!

uwsgi_response_writev_headers_and_body_do(): Broken pipe
    [core/writer.c line 287] during GET _URL_ (XX.XX.XX.XX)

OSError: write error

It seems that uWSGI tries to write in a stream but this stream has already been closed. When I check nginx log (error.log):

upstream prematurely closed connection while reading response
    header from upstream ...

Of course, my client (REST client or browser) receives a 502 error.

I always get this error after ~4s.

However, I don't know how to prevent this issue. I tried to set some parameters in my nginx config file:

location my_api_url {
    [...]
    uwsgi_buffer_size 32k;
    uwsgi_buffers 8 32k;
    uwsgi_busy_buffers_size 32k;

    uwsgi_read_timeout 300;
    uwsgi_send_timeout 300;

    uwsgi_connect_timeout 60;
}

But the issue is still here. I also tried to set these parameters in the uWSGI configuration file (wsgi.ini):

buffer-size=8192
ignore-sigpipe=true
ignore-write-errors=true

Before to try to optimize the response time, I hope this issue has a solution. I don't find one that's working in another post. I work with a large amount of data, so my response time, for some case, will be between 4-10s.

Hope you can help me :)

Thanks a lot in advance.

Answer 1

It may be the case that when you upload things, you use chunked encoding. There is uWSGI option --chunked-input-timeout, that by default is 4 seconds (it defaults to value of --socket-timeout, which is 4 seconds).

Though problem theoretically may lie somewhere else, I suggest you to try aforementioned options. Plus, annoying exceptions are the reason why I have

ignore-sigpipe=true
ignore-write-errors=true
disable-write-exception=true

in my uWSGI config (note that I provide 3 options, not 2):

• ignore-sigpipe makes uWSGI not show SIGPIPE errors;
• ignore-write-errors makes it not show errors with e.g. uwsgi_response_writev_headers_and_body_do;
• disable-write-exception prevents OSError generation on writes.

Answer 2

In my case, Nginx as a backproxy of uwsgi, the config http-timeout set the server to normally wait in long running requests.

Mind that the following options included in the nginx proxy declaration:

proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_send_timeout 300s;

were doing nothing regarding the gateway timeout.

Answer 3

I added [socked-timeout=xxx] in my uwsgi.ini config, it worked.