Mixing lightweight transactions and normal writes in Cassandra

Question

The Datastax documentation for lightweight transaction states:

"Lightweight transactions use a timestamping mechanism different than for normal operations and mixing LWTs and normal operations can result in errors. If lightweight transactions are used to write to a row within a partition, only lightweight transactions for both read and write operations should be used."

This is very vague (BTW what do they even mean by LWTs for read operations?).

Can someone with deeper knowledge of Cassandra internals elaborate on possible problems when mixing LWTs and normal writes?

My best guess is that there might be problems with concurrency (obviously). But I thought that if I (for example) insert a row using IF NOT EXISTS and if that was successful I later do UPDATE of the same partition, I will be fine. Am I wrong?

Answer 1

I agree that the documentation is not quite accurate on LWT constraints so let me clarify them:

• If you use INSERT ... IF NOT EXISTS, all deletes should also use DELETE ... IF EXISTS
• If you use UPDATE table SET column1=val1 WHERE <filters> IF column2=val2 then all updates/inserts on column1 & column2 should use LWT too. A common usage is when column1 and column2 are the same column.

Indeed LWT has 4 rounds:

1. Propose a Paxos ballot (a timeuuid value) to all replicas of the given partition

2. Check the condition (IF NOT EXISTS, IF EXISTS or IF col=val). If the condition is not met, fail here

3. Else, wait for a QUORUM/LOCAL_QUORUM to accept the Paxos ballot

4. Commit and apply the mutation (INSERT, UPDATE or DELETE)

Step b and c can be seen as a Compare And Swap.

The guarantees are that all writes using LWT are linearizable with respect to each others, so they are atomic and isolated on the partition.

Now if you use INSERT ... IF NOT EXISTS and then simple DELETE without LWT, you defeat the purpose of LWT and the LWT guarantees no longer apply.

Since the Compare phase (step 2) and the Commit phase (step 4) are in different steps, the only way to provide atomicity is to force others mutations to go throught LWT otherwise the logic is broken.

Same remark for the conditional updates with LWT.