I have a model that is sorted in a particular order. My goal is to find a record from the model where the sum of a particular column of all previous records equals a certain number. The following example gets me what I need, but it is very slow, especially with a rather large table. Are there any faster ways to solve for the product.id where the sum of all previous products' points = 100000?
total_points = 0
find_point_level = 100000
@products = Product.order("id").all
@products.each do |product|
total_points = product.points + total_points
@find_product = product.id
break if total_points >= find_point_level
end
Update
Here are some times for a few of the solutions below. This is going through about 60,000 records. Times are for ActiveRecord.
Original example (above):
2685.0ms
1238.8ms
1428.0ms
Original example using find_each:
799.6ms
799.4ms
797.8ms
Creating a new column with the sums:
181.3ms
170.7ms
172.2ms
You can avoid most n+1 queries in rails by simply eager loading associations. Eager loading allows you to load all of your associations (parent and children) once instead of n+1 times (which often happens with lazy loading, rails' default). As seen above, . includes allows nested association eager loading!
The additional difference between find() and find_by() is that find could only be used to search by primary key (usually the 'id') while the find_by() requires and searches by attribute (either passed as hash like Employee. find_by(name: 'Mike') or using the Employee.
The n+1 query problem is one of the most common scalability bottlenecks. It involves fetching a list of resources from a database that includes other associated resources within them. This means that we might have to query for the associated resources separately.
You can try denormalizing your database, and keeping partial sums directly in products
table. Simple query with where
and limit
would return you a proper answer in no time.
You need to create additional filter, that will update single record whenever product is added, and all products whenever product is deleted or it's points
field were changed.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With