Transactions and watch statement in Redis

Question

Could you please explain me following example from "The Little Redis Book":

With the code above, we wouldn't be able to implement our own incr command since they are all executed together once exec is called. From code, we can't do:

redis.multi()  current = redis.get('powerlevel')  redis.set('powerlevel', current + 1)  redis.exec() 

That isn't how Redis transactions work. But, if we add a watch to powerlevel, we can do:

redis.watch('powerlevel')  current = redis.get('powerlevel')  redis.multi()  redis.set('powerlevel', current + 1)  redis.exec() 

If another client changes the value of powerlevel after we've called watch on it, our transaction will fail. If no client changes the value, the set will work. We can execute this code in a loop until it works.

Why we can't execute increment in transaction that can't be interrupted by other command? Why we need to iterate instead and wait until nobody changes value before transaction starts?

Answer 1

There are several questions here.

1) Why we can't execute increment in transaction that can't be interrupted by other command?

Please note first that Redis "transactions" are completely different than what most people think transactions are in classical DBMS.

# Does not work redis.multi()  current = redis.get('powerlevel')  redis.set('powerlevel', current + 1)  redis.exec() 

You need to understand what is executed on server-side (in Redis), and what is executed on client-side (in your script). In the above code, the GET and SET commands will be executed on Redis side, but assignment to current and calculation of current + 1 are supposed to be executed on client side.

To guarantee atomicity, a MULTI/EXEC block delays the execution of Redis commands until the exec. So the client will only pile up the GET and SET commands in memory, and execute them in one shot and atomically in the end. Of course, the attempt to assign current to the result of GET and incrementation will occur well before. Actually the redis.get method will only return the string "QUEUED" to signal the command is delayed, and the incrementation will not work.

In MULTI/EXEC blocks you can only use commands whose parameters can be fully known before the begining of the block. You may want to read the documentation for more information.

2) Why we need to iterate instead and wait until nobody changes value before transaction starts?

This is an example of concurrent optimistic pattern.

If we used no WATCH/MULTI/EXEC, we would have a potential race condition:

# Initial arbitrary value powerlevel = 10 session A: GET powerlevel -> 10 session B: GET powerlevel -> 10 session A: current = 10 + 1 session B: current = 10 + 1 session A: SET powerlevel 11 session B: SET powerlevel 11 # In the end we have 11 instead of 12 -> wrong 

Now let's add a WATCH/MULTI/EXEC block. With a WATCH clause, the commands between MULTI and EXEC are executed only if the value has not changed.

# Initial arbitrary value powerlevel = 10 session A: WATCH powerlevel session B: WATCH powerlevel session A: GET powerlevel -> 10 session B: GET powerlevel -> 10 session A: current = 10 + 1 session B: current = 10 + 1 session A: MULTI session B: MULTI session A: SET powerlevel 11 -> QUEUED session B: SET powerlevel 11 -> QUEUED session A: EXEC -> success! powerlevel is now 11 session B: EXEC -> failure, because powerlevel has changed and was watched # In the end, we have 11, and session B knows it has to attempt the transaction again # Hopefully, it will work fine this time. 

So you do not have to iterate to wait until nobody changes the value, but rather to attempt the operation again and again until Redis is sure the values are consistent and signals it is successful.

In most cases, if the "transactions" are fast enough and the probability to have contention is low, the updates are very efficient. Now, if there is contention, some extra operations will have to be done for some "transactions" (due to the iteration and retries). But the data will always be consistent and no locking is required.