Delayed_job vs. Appoxy SimpleWorker

Question

I was about to start getting delayed_job up and running on my app when I found Appoxy SimpleWorker on Heroku.

Appoxy says it's massively parallel and that it scales workers up and down to minimize costs. However, it seems to me that with HireFire, delayed_job is as well. Here's my question: which would you pick: delayed_job + HireFire or SimpleWorker? (and why?)

https://github.com/tobi/delayed_job
http://hirefireapp.com/
http://addons.heroku.com/simple_worker

My app is small at the moment - the volume is low and the number of apps shouldn't be extraordinarily high for a good while.

---

Where else could one ever expect to get replies directly from the Founders of both services? Thanks Michael and Travis!

Answer 1

Hi (this is the author of HireFire)

I'll try to provide some information regarding the differences between the services. Not sure if it'll help much with your decision, but at least it's something!

Both solutions take a different approach. HireFire merely scales your Heroku web and worker dynos when necessary. You don't have to change anything in your existing code-base, you just use Delayed Job as normal. You don't have to send / write code for a separate environment/platform since it's compiled as a slug on Heroku's platform upon deployment, and when a new web or worker dyno spins up, the slug is used and runs immediately (containing all your ENV variables / settings.

The downside of HireFire as compared to SimpleWorker is that HireFire is Heroku-specific. So if you ever switch from Heroku to for example EngineYard or a VPS/Dedicated box, then HireFire won't work, but SimpleWorker will since it's not strictly tied to Heroku. Although, it's probably very cheap to host on a non-PaaS platform (in comparison) so auto-scaling wouldn't necessarily be needed as much or at all.

I have been a customer of SimpleWorker prior to developing HireFire and the thing I personally disliked was that I had to push part of my code-base to SimpleWorker, and load in my Rails environment, re-connect to the database from their servers location and also do API requests(?) every time I want to send a job to the cloud (although that might've changed now, so I encourage you to check it out for yourself to be sure, and maybe it isn't a big of an issue for you as it was for me). For me it was simply too much hassle/trial and error every time I wanted to add new job classes and had to load in all the individual pieces of code from my app and my gems in to the job class files itself, whereas running heroku ps:workers 1 or heroku ps:scale worker=2 would instantly spin up a worker or two and start processing with zero modifications to my code base, exactly the same as when I run it locally, since my whole app is already compiled as a slug on Heroku it just uses that, including my ENV variables and other settings/addons, and it spins up fast.

With HireFire you simply need to add the hirefireapp gem to your Gemfile, add your Heroku account/applications to the HireFire web interface, customize your scaling needs, deploy your application to Heroku, and that's it. Your applications will be constantly monitored and managed/adjusted (by the minute or less).

HireFire doesn't come with a slick interface with tables of jobs and their status (running/finished/failed/etc) (it does have an overview of the current amount of web/worker dynos and queued jobs for each app of course, and customizable scaling settings), although that's really the job of the worker library to provide such functionality. Delayed Job from what I know has one or two little admin interfaces you could use (open source) which are not tied to HireFire. Because SimpleWorker it both a hosted service, as well as a worker library in one, they provide you with a web interface as well.

HireFire has the ability to scale your web dynos as well, not just your worker dynos.

Both services have the ability to process a lot of jobs in parallel, since both Heroku and SimpleWorker are pro-rated to the second from what I understand. So whether you spin up 10 worker dynos for 6 seconds, or 1 for 60 seconds makes no difference (or barely) in costs.

I haven't used SimpleWorker after releasing HireFire myself, which was quite a while ago so I'm not sure what else SimpleWorker provides these days or if they simplified the process since then, so I'm not sure whether the above statements I made are still valid at this time.

Hope this helps!

Answer 2

Hi (founder of SimpleWorker),

Michael summed it up quite nicely, but I'll add a few differences between Heroku Workers and SimpleWorker, not necessarily HireFire.

• Heroku has a maximum of 24 workers at one time which means only 24 jobs can run at once. With SimpleWorker, you can run 1000's at once (and more as we grow) without any changes.
• Scaling with SimpleWorker requires no effort so as your application grows, you don't need to change anything, just keep throwing more jobs our way.
• Heroku charges whether you are using the workers or not, SimpleWorker does not. This is the problem HireFire solves though so it's not an issue if you're using HireFire.
• SimpleWorker has scheduling built in so you don't need cron or anything to kick off your work.
• Management interface so you can visualize all your workers/jobs, find errors, get error alerts, view logs for all your jobs, etc.

The downside is that it takes a bit more thought to use SimpleWorker since it's not running your full Rails environment. You have to think of your workers as separate entities running on a different system (which they are). For simple things like sending an email in the background or offloading some things from your front end here and there, Heroku Workers are probably a good choice. If you do any large batch jobs, scheduling, long running jobs, want more visibility into your jobs, etc., then you might want to give SimpleWorker a try.

Btw, we recently put up some new videos so you can get an idea of how it works and how easy it is to use: http://www.simpleworker.com/how_it_works/videos

Hope that helps! And feel free to ask me any questions you have about SimpleWorker.