batch destroy all in rails active record

Question

I have a large number of Product objects, that all have an associated "variant" with them. I can't just delete_all products because I need the association to be deleted as well.

When I use Products.all.destroy_all it freezes and doesn't finish destroy all the products and then it forces me to cancel it and roll it back.

Is there a solution to this?

Answer 1

For large number of objects and associated 'variant' with them, you have to use destroy_all.

Product.all.destroy_all

Since destroy_all actually loads the entire relation and then iteratively destroys the records one by one, you can blow your memory gasket very easily. So let's do the right thing by default and do this work in batches of 100 by default and allow you to specify the batch size like so: destroy_all(batch_size: 100).

Product.destroy_all(batch_size: 100)

OR

You can use in_batches also

Product.in_batches(of: 200).destroy_all

Answer 2

If "variant" records don’t have any dependencies that would have to be deleted from the database use dependent: :delete_all instead of dependent: :destroy

products_count = Product.count

# Determine how many batches need to be run
number_of_iterations = (products_count.to_f / 1000).ceil

(1..number_of_iterations).each do |i|
  Product.limit(1000).delete_all
end

When dealing with MASSIVE amounts of data it’s good to batch the deletion. If you delete more than 5,000 rows in a single transaction, your database will lock. This means the entire table is inaccessible by any other running process for the duration of the transaction. This can mean some serious issues for the users of your site while a DELETE is happening.