Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Redis: How to intersect a "normal" set with a sorted set?

Tags:

Assume I have a set (or sorted set or list if that would be better) A of 100 to 1000 strings.

Then I have a sorted set B of many more strings, say one million.

Now C should be the intersection of A and B (of the strings of course).

I want to have every tuple (X, SCORE_OF_X_IN_B) where X is in C.

Any Idea?

I got two ideas:

  1. Interstore
    • store A a sorted set with every score being 0
    • interstore to D
    • get every item of D
    • delete D
  2. Simple loop in client
    • loop over A in my client programm
    • get zscore for every string

While 1. has way too much overhead on the redis side (Has to write for example. The redis page states quite a high time complexity, too http://redis.io/commands/zinterstore), 2. would have |A| database connections and won't be a good choice.

Maybe I could write a redis/lua script which will work like zscore but with an arbitrary number of strings, but I'm not sure if my hoster allows scripts...

So I just wanted to ask SO, if there is an elegant and fast solution available without scripting!

like image 364
user562529 Avatar asked May 08 '12 14:05

user562529


People also ask

How do you find the intersection of two sets in Redis?

Redis and PHPRedis ZINTERSTORE command computes the intersection of numkeys sorted sets given by the specified keys, and stores the result in the destination. It is mandatory to provide the number of input keys (numkeys) before passing the input keys and the other (optional) arguments.

Is Redis set sorted?

Redis lists are lists of strings sorted by insertion order.

How does Redis sorted sets work?

In Redis, sorted sets are a data type similar to sets in that both are non repeating groups of strings. The difference is that each member of a sorted set is associated with a score, allowing them to be sorted from the smallest score to the largest.

How is Redis sorted set implemented?

In redis documentation is said that: "Sorted sets are implemented via a dual-ported data structure containing both a skip list and a hash table, so every time we add an element Redis performs an O(log(N)) operation".


1 Answers

There is a simple solution to your problem: ZINTERSTORE will work with a SET and a ZSET. Try:

redis> sadd foo a (integer) 1 redis> zadd bar 1 a (integer) 1 redis> zadd bar 2 b (integer) 1 redis> zinterstore baz 2 foo bar AGGREGATE MAX (integer) 1 redis> zrange baz 0 -1 withscores 1) "a" 2) "1" 

Edit: I added AGGREGATE MAX above, since redis will give each member of the (non-sorted) set foo a default score of 1, and SUM that with whatever score it has in the (sorted) set bar.

like image 139
Linus Thiel Avatar answered Nov 07 '22 21:11

Linus Thiel