How to configure nginx + Unicorn to avoid timeout errors?

Question

I have a Rails (v3.2.13, Ruby 2.0.0) application running on nginx + Unicorn (Ubuntu 12.04). All is working well, except when an admin user is uploading users (thousands) via a CVS file. The problem is that I have set timeout to 30 seconds and the import process takes much more time. So, after 30 seconds I get an nginx 502 Bad Gateway page (Unicorn worker is killed).

The obvious solution is to increase timeout, but I don't want this because it'll cause another problems (I guess), because it's not a typical behavior.

Is there a way to handle this kind of problems?

Thanks a lot in advance.

PS: Maybe a solutions is to modify the code. If so, I want to avoid the user to perform another request.

Some ideas (don't know if possible):

• Setup a worker dedicated to this request.
• Send a "work in progress" signal to Unicorn to avoid to be killed.

nginx-app.conf

upstream xxx {
  server unix:/tmp/xxx.socket fail_timeout=0;
}


server {
  listen   80; 

  ...

  location / {

    proxy_pass  http://xxx;
    proxy_redirect     off;
    ...

    proxy_connect_timeout      360;
    proxy_send_timeout         360;
    proxy_read_timeout         360;
  }
}

unicorn.rb

worker_processes 2

listen "/tmp/xxx.socket"

timeout 30

pid "/tmp/unicorn.xxx.pid"

Answer 1

This is a good reason to create a queue. And you will:

• upload csv file (that should be within 30sec)
• your background job that will import user data (that can go for hours…)
• while this job is in progress you can serve some kind of WIP page with job status/percents/etc.

Check https://github.com/resque/resque for example. There is a lot of other queues.

Answer 2

Is there a way to handle this kind of problems?

Do the job in background. You should have a separate process that gets jobs from queue one by one and processes them. And since it doesn't work with user requests, it can do its job as long as needed. You don't need unicorn for this, just a separate daemon.