Running command-line processes in parallel in Ruby

Question

I'm using PhantomJS, a command-line tool, to render images of websites, and I want to run a number of these in parallel instead of doing one after the other. How can I do this?

Answer 1

Here's an Example using Resque. Note I've left escaping out for brevity... you should never pass external inputs directly into shell commands.

class RasterizeWebPageJob
  @queue = :screenshots
  def self.perform(url)
    system("/usr/bin/env DISPLAY=:1 phantomjs rasterize.js #{url} ...")
  end
end

10.times { Resque.enqueue(RasterizeWebPageJob, "http://google.com/") }

Provided you're running enough workers (and there are workers available), they'll execute in parallel. The important thing here is that you put separate jobs onto the queue instead of processing multiple screenshots from within the one job.

I'd advise against using Thread.new in a Rails controller. Queues are much easier (and safer) to manage than Threads.