Menu
I have a delayed job queue which contains particularly slow running tasks, which I want to be crunched by its own set of dedicated workers, so there is less risk it'll bottleneck the rest of the worker pipeline.
RAILS_ENV=production script/delayed_job --queue=super_slow_stuff start
However I then also want a general worker pool for all other queues, hopefully without having to specify them seperately (as their names etc are often changed/added too). Something akin to:
RAILS_ENV=production script/delayed_job --except-queue=super_slow_stuff start
I could use the wildcard * charecter but I imagine this would cause the second worker to pickup the super slow jobs too?
Any suggestions on this?
332
Delayed::Job works by serializing a Ruby object as YAML and storing it in a database table so that a worker in a different process can retrieve, deserialize, and perform the requested work. Some operations that we use Delayed::Job for are longer-running data processing, sending emails, and updating search indicies.
Delayed Job, also known as DJ, makes it easy to add background tasks to your Rails applications on Heroku. You can also use Resque and many other popular background queueing libraries. Delayed Job uses your database as a queue to process background jobs.
Active Job is a framework for declaring jobs and making them run on a variety of queueing backends. These jobs can be everything from regularly scheduled clean-ups, to billing charges, to mailings. Anything that can be chopped up into small units of work and run in parallel, really.
The most simple way to check whether delayed_job is running or not, is to check at locked_by field. This field will contain the worker or process locking/processing the job. Running Delayed::Job. where('locked_by is not null') will give you some results, if there are jobs running.
you can define a global constant for your app with all queues.
QUEUES={
mailers: 'mailers',
etc..
}
then use this constant in yours delay method calls
object.delay(queue: QUEUES[:mailers]).do_something
and try to build delayed_job_args dinamically
system("RAILS_ENV=production script/delayed_job --pool=super_slow_stuff --pool:#{(QUEUES.values-[super_slow_stuff]).join(',')}:number_of_workers start")
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
