Calculate number of events per last 24 hours with Redis

Question

Seems, it's common task, but I haven't found solution.

I need to calculate number of user's events (for example, how many comments he left) per last 24 hours. Older data is not interesting for me, so information about comments added month ago should be removed from Redis.

Now I see only one solution. We can make keys that includes ID of user and hour of day, increment it value. Then we will get 24 values and calculate their sum. Each key has 24 hours expiration.

For example,

1. Event at Jun 22, 13:27 -> creating key _22_13 = 1
2. Event at Jun 22, 13:40 -> incrementing key _22_13 = 2
3. Event at Jun 22, 18:45 -> creating key _22_18 = 1
4. Event at Jun 23, 16:00 -> creating key _23_16 = 1
5. Getting sum of event at Jun 23, 16:02 -> sum of keys _22_17 - _22_23, _23_00 - _23_16: in our keys it's only _22_18 and _23_16, result is 2. Keys _22_13 and _22_13 are expired.

This method is not accurate (it calculates count of events per 24+0..1 hour) and not so universal (what key will I choose if I need number of events per last 768 minutes or 2.5 month?).

Do you have better solution with Redis data types?

Answer 1

Your model seems fine. Of course, it's not universal, but it's what you have to sacrifice in order to be performant.

I suggest you keep doing this. If you will need another timespan for reporting (768 minutes), you can always get this from mysql where your comments are stored (you do this, right?). This will be slower, but at least the query will be served.

If you need faster response or higher precision, you can store counters in redis with minute resolution (one counter per minute).