Delayed job exclude queue

Question

I have a delayed job queue which contains particularly slow running tasks, which I want to be crunched by its own set of dedicated workers, so there is less risk it'll bottleneck the rest of the worker pipeline.

RAILS_ENV=production script/delayed_job --queue=super_slow_stuff start

However I then also want a general worker pool for all other queues, hopefully without having to specify them seperately (as their names etc are often changed/added too). Something akin to:

RAILS_ENV=production script/delayed_job --except-queue=super_slow_stuff start

I could use the wildcard * charecter but I imagine this would cause the second worker to pickup the super slow jobs too?

Any suggestions on this?

Answer 1

you can define a global constant for your app with all queues.

QUEUES={
  mailers: 'mailers',
  etc..
}

then use this constant in yours delay method calls

object.delay(queue: QUEUES[:mailers]).do_something

and try to build delayed_job_args dinamically

system("RAILS_ENV=production script/delayed_job --pool=super_slow_stuff --pool:#{(QUEUES.values-[super_slow_stuff]).join(',')}:number_of_workers start")