Delayed Job Batch Inserts

Question

I'm in the process of optimizing my SQL queries on my heroku server so I can speed things up on one particular request. Right now I'm mainly looking at condensing all the INSERT queries into the fewest queries as possible.

At some point in my code I have this:

  jobs.each do |j|
    Delayed::Job.enqueue j
  end

I found out that every iteration sends a BEGIN, INSERT, COMMIT to the db. That jobs array can have from a few to a couple hundred objects in it. I have looked for a way to batch insert delayed jobs but couldn't find anything. Any idea of how to achieve that?

Answer 1

I've been using AR-Extensions for a long time to insert bulk data from models into the database.

That was on Rails 2.3.x though, be careful that there are now different versions depending on the Rails version: http://www.continuousthinking.com/tags/arext

I'm not sure how Delayed::Job works, but guessing from your example, I'd assume it inserts a record per job into a table which then serves as the queue. You could extend that, using AR-Extensions, to collect all those models and insert all jobs at once.

Answer 2

I ended up enqueuing my User object instead, which had a jobs attribute. So 1 insert instead of jobs.length inserts.