How to find out why uWSGI kill workers?

Question

i have app on Pyramid. I run it in uWSGI with these config:

[uwsgi]
socket = mysite:8055
master = true
processes = 4
vacuum = true
lazy-apps = true
gevent = 100

And nginx config:

server {
    listen 8050;
    include uwsgi_params;

    location / {
        uwsgi_pass mysite:8055;
    }
}

Usually all fine, but sometimes uWSGI kills workers. And i have no idea why.

I see in uWSGI logs:

DAMN ! worker 2 (pid: 4247) died, killed by signal 9 :( trying respawn ...
Respawned uWSGI worker 2 (new pid: 4457)

but in the logs there is no Python exceptions.

sometimes i see in uWSGI logs:

invalid request block size: 11484 (max 4096)...skip
[uwsgi-http key: my site:8050 client_addr: 127.0.0.1 client_port: 63367] hr_instance_read(): Connection reset by peer [plugins/http/http.c line 614]

And nginx errors.log:

*13388 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1,
*13955 recv() failed (104: Connection reset by peer) while reading response header from upstream, client:

I think this can be solved by adding buffer-size=32768, but it is unlikely due to this uWSGI kill workers.

Why uwsgi can kill workers? And how can I know the reason? The line "DAMN ! worker 2 (pid: 4247) died, ..." nothing to tells.

Answer 1

signal 9 means it received a SIGKILL. so something sent a kill to your worker. it's relatively likely that the out-of-memory killer decided to kill your app because it was using too much memory. try watching the workers with a process monitor and see if it uses a lot of memory.