Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Pop multiple values from Redis data structure atomically?

Is there a Redis data structure, which would allow atomic operation of popping (get+remove) multiple elements, which it contains?

There are well known SPOP or RPOP, but they always return a single value. Therefore, when I need first N values from set/list, I need to call the command N-times, which is expensive. Let's say the set/list contains millions of items. Is there anything like SPOPM "setName" 1000, which would return and remove 1000 random items from set or RPOPM "listName" 1000, which would return 1000 right-most items from list?

I know there are commands like SRANDMEMBER and LRANGE, but they do not remove the items from the data structure. They can be deleted separately. However, if there are more clients reading from the same data structure, some items can be read more than once and some can be deleted without reading! Therefore, atomicity is what my question is about.

Also, I am fine if the time complexity for such operation is more expensive. I doubt it will be more expensive than issuing N (let's say 1000, N from the previous example) separate requests to Redis server.

I also know about separate transaction support. However, this sentence from Redis docs discourages me from using it for parallel processes modifying the set (destructively reading from it):
When using WATCH, EXEC will execute commands only if the watched keys were not modified, allowing for a check-and-set mechanism.

like image 833
Pavel S. Avatar asked Dec 16 '13 21:12

Pavel S.


People also ask

Is Redis LPOP Atomic?

Is Redis LPOP / RPOP operation atomic? Yes, both LPOP and RPOP are atomic.

Is Redis Lrange Atomic?

And, yes, this command is atomic, so no two clients will receive the same elements.

What is LPOP in Redis?

Removes and returns the first elements of the list stored at key . By default, the command pops a single element from the beginning of the list. When provided with the optional count argument, the reply will consist of up to count elements, depending on the list's length.

What is WATCH in Redis?

WATCH is used to provide a check-and-set (CAS) behavior to Redis transactions. WATCH ed keys are monitored in order to detect changes against them. If at least one watched key is modified before the EXEC command, the whole transaction aborts, and EXEC returns a Null reply to notify that the transaction failed.


1 Answers

Use LRANGE with LTRIM in a pipeline. The pipeline will be run as one atomic transaction. Your worry above about WATCH, EXEC will not be applicable here because you are running the LRANGE and LTRIM as one transaction without the ability for any other transactions from any other clients to come between them. Try it out.

like image 73
Eli Avatar answered Sep 22 '22 16:09

Eli