How to limit count of items in Redis sorted sets

Question

In my case I upload a lot of records to Redis sorted set, but I need to store only 10 highest scored items. I don't have ability to influence on the data which is uploaded (to sort and to limit it before uploading).

Currently I just execute

ZREMRANGEBYRANK key 0 -11

after uploading finish, but such approach looks not very optimal (it's slow and it will be better if Redis could handle that).

So does Redis provide something out of the box to limit count of items in sorted sets?

Answer 1

Nopes, redis does not provide any such functionality apart from ZREMRANGEBYRANK .

There is a similar problem about keeping a redis list of constant size, say 10 elements only when elements are being pushed from left using LPUSH.

The solution lies in optimizing the pruning operation.

Truncate your sorted set, once a while, not everytime

Methods:

1. Run a ZREMRANGEBYRANK with 1/5 probability everytime, using a random integer.

2. Use redis pipeline or Lua scripting to achieve this , this would even save the two network calls happening at almost every 5th call.

This is optimal enough for the purpose mentioned.

Algorithm example:

ZADD key member1 score1
random_int = some random number between 1-5
if random_int == 1:  # trim sorted set with 1/5 chance
   ZREMRANGEBYRANK key 0 -11