Redis cluster and zsets

Question

In client side sharding approach I can create a shard and store a zset in that single shard. With Redis cluster, how big can a zset be? How do I scale zsets if is going to be an index of all the keys I have in redis.

Answer 1

Short answer: yes - you can store all your key names in a single Sorted Set.

A Redis Sorted Set can have up to 2^32 - 1 (4294967295) members, each being made up of a string up to 512MB long and a 64bit score (the same limit goes for the number of keys in the database). This applies both to standalone Redis as well as Redis Cluster and allows storing ~2048PB in a single Sorted Set :)

In Redis Cluster, every key belongs to a specific hash slot. Therefore, a Sorted Set, regardless the theoretical size limits above, can only be as big as your biggest shard (i.e. Redis server). While a shard can manage a single slot, and a slot can hold a single key - I hardly recommend that approach due to scalability concerns.

Indexing all your keys is an interesting challenge - why are you doing that? And more importantly, what types of queries do you want to run against that index? Depending on the number of keys in your database and querying requirements, there may be other, more efficient ways to achieve that.