Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Rails build and save in bulk

I've got something like the following:

module Bar < ActiveRecord::Base
  belongs_to :foo
  ...

module Foo < ActiveRecord::Base
  has_many :bars, dependent: :destroy

  def build_bars
    1000.times do |i|
      bars.build(num: i)
    end
  end

  def create_default_bars!
    build_bars
    save
  end

Notice that Foo#build_bars is cheap. Even though it loops 1000 times, it takes very little time. But then, once you hit save, suddenly ActiveRecord decides to perform 1000 inserts, which is incredibly slow.

How can I write a custom save_the_bars method such that it performs a single bulk INSERT query for all of the bars I have built up (I remind you, the 1000 builds are apparently very cheap), but which functions as a drop-in replacement for save in this example?


I'm expecting an answer along the lines of recommendation #3 of this blog post:

https://www.coffeepowered.net/2009/01/23/mass-inserting-data-in-rails-without-killing-your-performance/

But since my example uses build and relies on some slightly nontrivial rails magic, it isn't immediately obvious how to translate it over. Bonus points for benchmarks!

like image 582
Dan Burton Avatar asked Sep 21 '25 09:09

Dan Burton


1 Answers

I would try the activerecord-import gem.

def save_the_bars(bars)
  Bars.import bars
end

This call to import does whatever is most efficient for the underlying database adapter. Pretty slick, eh?

For bonus points: Benchmarks


Question-asker here, hijacking this answer with details on what I did following the above suggestion:

def build_bars
  built_bars = []
  1000.times do |i|
    built_bars << bars.build(num: i)
  end
  built_bars
end

def create_default_bars
  save
  Bar.insert built_bars, validate: false
  reload
end

This gave a nice speedup for relatively little effort. I still suspect that more speedup could be had (by leveraging the nuances of insert) but I'm satisfied with this for now. In my use case, it was safe to turn off validation, since the method generating all the Bars is guaranteed to generate valid ones.

like image 196
Dan Grahn Avatar answered Sep 23 '25 01:09

Dan Grahn