Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

In Rails 3.2, how to "pluck_in_batches" for a very large table

I have a massive table Foo from which I need to pluck all values in a certain field, Foo.who.

The array has millions of rows, but only a few thousand different values in the who column.

If the table was smaller of course I'd simply use Foo.pluck(:who)

If I use Foo.find_in_batches do |a_batch| each set is an Array of Foo records, rather than an activerecord collection of Foo records, so I cannot use .pluck() and AFAIK the only way to extract the who column is via something like .map(&:who) that iterates over the array.

Is there a way to pluck the who column from Foo in batches that does not require then iterating over each element of each batch to extract the who column?

like image 245
jpw Avatar asked Feb 08 '15 06:02

jpw


1 Answers

In Rails 5 you can use:

Foo.in_batches do |relation|
  values = relation.pluck(:id, :name, description)
  ...
end

Upd: for prevent memory leaks use:

Foo.uncached do
  Foo.in_batches do |relation|
    values = relation.pluck(:id, :name, description)
    ...
  end
end
like image 76
Dmitry Ukolov Avatar answered Oct 01 '22 22:10

Dmitry Ukolov