Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

storing time series in redis

I need to store some timeseries data into redis. I have unix timestamps and that that i need to associate a value (timestamp <-> value)

I tried sorted sets having the timestamp as score (so i can do zrange on the timestamps) and the value as member.

127.0.0.1:6379> ZADD timeserie 1392141527245 10 1392141527275 12 1392141527100 10
(integer) 2
127.0.0.1:6379> zscan timeserie 0
1) "0"
2) 1) "10"
   2) "1392141527245"
   3) "12"
   4) "1392141527275"
127.0.0.1:6379>

But i hit a problem, members are nonrepeating while my values can be the same for different timestamps. Any idea how to approach this? Is another data type better?

like image 409
maephisto Avatar asked Feb 21 '14 11:02

maephisto


People also ask

Is Redis a time series database?

Redis has been used for storing and analyzing time series data since its creation.

Does Redis store timestamp?

Redis doesn't store this information.

What is the best way to store time series data?

Storing time series data. Time series data is best stored in a time series database (TSDB) built specifically for handling metrics and events that are time-stamped. This is because time series data is often ingested in massive volumes that require a purpose-built database designed to handle that scale.

Is Redis good for long term storage?

The only difference from the redis service is that it is configured to store data permanently rather than toss data out when it runs out of memory (as a cache configuration would do). That also means data stored in Redis is replicated when an environment is branched, just like for MySQL, Elasticsearch, or MongoDB.


2 Answers

An easy trick to solve this problem is to concatenate the timestamp and the value.

Instead of storing:

ZADD timeserie 1392141527245 10

you can store:

ZADD timeserie 1392141527245 10:1392141527245

Up to the application to encode/parse the value:timestamp format.

Sorted sets are implemented as a skip list plus a hash table, so they are not especially compact in memory. If the volume of your data is significant, you will be better served by another solution.

Some people use normal strings to encode time series, which are way more compact than sorted sets. You can find an example here: https://github.com/antirez/redis-timeseries

like image 110
Didier Spezia Avatar answered Nov 04 '22 10:11

Didier Spezia


It's a sorted set, so indeed value can't repeat. Redis might not be the best tool for your usage, give a try to a specialised database like influxdb.

like image 34
FGRibreau Avatar answered Nov 04 '22 08:11

FGRibreau