I need to store some timeseries data into redis. I have unix timestamps and that that i need to associate a value (timestamp <-> value)
I tried sorted sets having the timestamp as score (so i can do zrange on the timestamps) and the value as member.
127.0.0.1:6379> ZADD timeserie 1392141527245 10 1392141527275 12 1392141527100 10
(integer) 2
127.0.0.1:6379> zscan timeserie 0
1) "0"
2) 1) "10"
2) "1392141527245"
3) "12"
4) "1392141527275"
127.0.0.1:6379>
But i hit a problem, members are nonrepeating while my values can be the same for different timestamps. Any idea how to approach this? Is another data type better?
Redis has been used for storing and analyzing time series data since its creation.
Redis doesn't store this information.
Storing time series data. Time series data is best stored in a time series database (TSDB) built specifically for handling metrics and events that are time-stamped. This is because time series data is often ingested in massive volumes that require a purpose-built database designed to handle that scale.
The only difference from the redis service is that it is configured to store data permanently rather than toss data out when it runs out of memory (as a cache configuration would do). That also means data stored in Redis is replicated when an environment is branched, just like for MySQL, Elasticsearch, or MongoDB.
An easy trick to solve this problem is to concatenate the timestamp and the value.
Instead of storing:
ZADD timeserie 1392141527245 10
you can store:
ZADD timeserie 1392141527245 10:1392141527245
Up to the application to encode/parse the value:timestamp format.
Sorted sets are implemented as a skip list plus a hash table, so they are not especially compact in memory. If the volume of your data is significant, you will be better served by another solution.
Some people use normal strings to encode time series, which are way more compact than sorted sets. You can find an example here: https://github.com/antirez/redis-timeseries
It's a sorted set, so indeed value can't repeat. Redis might not be the best tool for your usage, give a try to a specialised database like influxdb.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With