Should I have a heroku worker dyno for poll a AWS SQS?

Question

Im confusing about where should I have a script polling an Aws Sqs inside a Rails application.

If I use a thread inside the web app probably it will use cpu cycles to listen this queue forever and then affecting performance.

And if I reserve a single heroku worker dyno it costs $34.50 per month. It makes sense to pay this price for it for a single queue poll? Or it's not the case to use a worker for it?

The script code:

What it does: Listen to converted pdfs. Gets the responde and creates the object into a postgres database.

  queue = AWS::SQS::Queue.new(SQSADDR['my_queue'])    
  queue.poll do |msg|
     ...
     id = received_message['document_id']
     @document = Document.find(id)
     @document.converted_at = Time.now
     ...
  end

I need help!! Thanks

Answer 1

You have three basic options:

1. Do background work as part of a worker dyno. This is the easiest, most straightforward option because it's the thing that's most appropriate. Your web processes handle incoming HTTP requests, and your worker process handles the SQS messages. Done.
2. Do background work as part of your web dyno. This might mean spinning up another thread (and dealing with the issues that can cause in Rails), or it might mean forking a subprocess to do background processing. Whatever happens, bear in mind the 512 MB limit of RAM consumed by a dyno, and since I'm assuming you have only one web dyno, be aware that dyno idling means your app likely isn't running 24x7. Also, this option smells bad because it's generally against the spirit of the 12-factor app.
3. Do background work as one-off processes. Make e.g. a rake handle_sqs task that processes the queue and exits once it's empty. Heroku Scheduler is ideal: have it run once every 20 minutes or something. You'll pay for the one-off dyno for as long as it runs, but since that's only a few seconds if the queue is empty, it costs less than an always-on worker. Alternately, your web app could use the Heroku API to launch a one-off process, programmatically running the equivalent heroku run rake handle_sqs.