Redis data structure design for sorting time-based values

Question

I'm performing some analysis on a data stream and publishing the results on a Redis channel. Consumers subscribe to these channels and get real-time data feeds. All historical data analysis results are lost.

Now I want to add the ability to store historical data in Redis so that consumers can query this historical data (mainly by time). Since the analysis results are partitioned by time what would be the a good design to store the results in Redis?

Answer 1

Use redis sorted sets.

Sorted sets store data based on "scores", so in your case, just use a time stamp in millis; the data will be sorted automatically, allowing you to retrieve historical items using start/end date ranges, here's an example...

Add items to a sorted set...

zadd historical <timestamp> <dataValue>

..add some sample data..

 zadd historical 1 data1
 zadd historical 2 data2
 zadd historical 3 data3
 zadd historical 4 data4
 zadd historical 5 data5
 zadd historical 6 data6
 zadd historical 7 data7

..retrieve a subset of items using start/end range...

 zrangebyscore historical 2 5

..returns...

1) "data2"
2) "data3"
3) "data4"
4) "data5"

So, in your case, if you want to retrieve all historical items for the last day, just do this...

zrangebyscore historical <currentTimeInMillis - 86400000> <currentTimeInMillis>