SSCAN vs SMEMBERS resources usage for large sets

Question

I have a large redis SET (more than 6M entries), and i need to go through all the entries and make some other redis operations with each (mostly calling a ZCARD in a redis Sorted Set using a key based on the original entry).

Which is the most efficient (in terms of resources) way of going through all the entries of the SET? Using SSCAN or doing an SMEMEBERS call.

Answer 1

SMEMBERS returns all members in a SET, in one operation. The duration of this operation is directly proportional to the number of items in the SET (time complexity: O(N) ).

During this operation, your instance won't respond to any other request.

SSCAN allows you to iterate through all items in a SET. The time complexity is fixed ( O(1) ), depending on the number of items you get at each call (this number is defined by the COUNT parameter). The total cost for SSCAN will probably be of the same order than for SMEMBERS, or maybe greater because you will have to make multiple calls. But it will allow other requests being processed between two calls, so your Redis instance won't look unresponsive.

All of this is pure theory. To get a definitive advice, you should test and measure, it should be rather easy to do.

Answer 2

SMEMBERS has to iterate over the whole set and produce the whole list of keys at once, while SSCAN returns a fraction of the keys each time you call it. While they both do pretty much the same amount of work, SSCAN does the work in smaller chunks which means, for most use cases, that it has a smaller effect on the performance of everything else the server is doing.