Rails storing aggregated information

Question

I'm making a Rails app which will feature a lot of "calculated" or "aggregated" data, i.e. information which is computed by performing expensive operations on the data stored by the user. I'm thinking I need some way of storing this data so A. I am not constantly performing expensive DB operations, and B. So that I can spit out "reports" with pretty graphs over time etc for given attributes.

I'm wondering what the best way to implement this is? I'd need to be calculating and storing values (numeric) for a given model, and how they change over time. I want this to be efficient and avoid duplicating and data. The records will be pretty much fixed once created, so I don't need to worry about things being changed too much, though it needs to be considered.

I just wondered what the most common approach to this is, and how I should go about implementing this within a Rails app?

Answer 1

I've been working on a similar problem and I've also worked on applications that did this the wrong way.

Here is my best-practices recommendation:

• Store the raw data in the model, let's call is Feed
• Set up a one-on-one association to another model that holds the computed values, e.g. FeedStats. This may also be a one-to-many association or many-to-one, depending on the exact case; you might roll up some individual Feed records into some sort of aggregate, etc.
• Keep all original raw data around. This will be extremely useful if you later want to switch the computation to some other algorithm and you may need to recompute old data, or if you discover bugs in the computation, etc.
• Set up the computation on a background task, using tools like Resque (with or without Scheduler), DelayedJob or similar.

If you can be a bit more specific and give some examples of your exact problem, I can perhaps give some more specific tips. Good luck.