Does anyone run more than one resque worker in a Heroku Dyno?

Question

Given that unicorn usually manages more than one Rails server process, and given that a Resque job runner probably consumes less resources than a Web request, it should be possible to run more than one resque worker on a single Heroku dyno.

Is anyone doing this successfully so far? My thoughts are, that an easy way to do so would have the Procfile runs foreman, which then runs 2 (or more) instances of the actual worker (i.e. rake resque:work)

Or is rake resque:workers up to that task? Resque itself does not recommend using that method, as this starts workers in parallel threads instead of in parallel processes.

Obviously, this makes sense only on i/o bound jobs.

Answer 1

One can use foreman to start multiple processes. Add foreman to your Gemfile, and then create two files:

Procfile:

worker: bundle exec foreman start -f Procfile.workers

Procfile.workers:

worker_1: QUEUE=* bundle exec rake resque:work
worker_2: QUEUE=* bundle exec rake resque:work

The same technique can be used to run a web server alongside some workers.

NOTE: while many state success using this approach, I would not suggest to use it outside of some experiments, mostly because of the risk to run into RAM limitations on small heroku instances; and once you pay for the heroku service it is probably easier to just spin up a dedicated worker machine anyways.

Answer 2

Based on this article, it sounds like it's possible, but the biggest gotcha is that if one of the child processes dies, Heroku won't be able to restart it.