Why is 2-phase commit not suitable for a microservices architecture?

Question

I've read a post saying that:

We can not implement traditional transaction system like 2 phase commit in micro-services in a distributed environment.

I agree completely with this.

But it would be great if someone here can explain the exact reason for this. What would be the issues I'm going to face if I'm implementing 2-phase commit with microservices?

Thanks in advance

Answer 1

The main reason for avoiding 2-phase commit is, the transaction co-ordinator is a kind of dictator as it tells all other nodes what to do. Usually the transaction co-ordinator is embedded in the application server. The problem happens when after the 1st phase or prepare phase the transaction co-ordinator or the application server goes down. Now, the participating nodes don't know what to do. They cannot commit because they don't know if others have replied to the co-ordinator with a "no" and they cannot rollback because others might have said a "yes" to the co-ordinator. So, until the co-ordinator comes back after 15 minutes (say) and completes the 2nd phase, the participating data stores will remain in a locked state. This inhibits scalability and performance. Worse things happen when the transaction log of the co-ordinator gets corrupted after the 1st phase. In that case, the data stores remain in the locked state forever. Even restarting the processes won't help. The only solution is to manually check the data to ensure consistancy and then remove the locks. These things usually happen in a high pressure situation and therefore it's definitely a huge operational overhead. Hence the traditional 2-phase commit is not a good solution.

However, it should be noted here that some of the modern systems like Kafka have also implemented a 2-phase commit. But this is different from the traditional solution in that here every broker can be a co-ordinator and thus the Kafka's leader election algorithm and the replication model alleviate the issues mentioned in the traditional model.

Answer 2

Some things to note and also give some background:

1. In most scenarios microservices interact via HTTP (a stateless protocol) and as a result global/ XA transactions are just not applicable/ possible.
2. Exactly once semantics are not possible and you should go for "at least once". This means all services should be idempotent.
3. One good example of why is not possible of achieving "exactly once" semantics in such a setup is that http connections very frequently are lost on the way back to the client. This means that via a POST the state of the server has changed, while the client receives a timeout error.
4. Inside the boundaries of a microservices you can use them just fine. As you mentioned Kafka you can quite easily consume (from 1 topic) and produce (to 1 or more topics) a single atomic/ all or nothing operation (exactly once semantics).
5. But if you want global and long running transactions among microservices that interact via http the only practical option (you might see global transaction via http if you google, but for a production system just ignore them), is to design for eventual consistency. In brief this means, retry for ever for recoverable errors (this is a whole chapter in itself) and expose compensating endpoints or produce compensating events that will eventually amend non-recoverable errors. Check out the sagas pattern. Narayana Transaction Manager has good Sagas support and a good products comparison.
6. Check out the related microservices patterns that offer an alternative to XA transactions (you might see this as global transactions or 2 phase commit/ 2PC) like Transactional Outbox or Event Sourcing that offer nice "at least once semantics".
7. Distributed systems are very complicated and you should have a reason to go for such a solution. If you go distributed, operations that your monolith can safely delegate to your transaction manager, will have to be dealt by the developer/ architect :-).
8. Also, the majority of non SQL databases/ systems do not support XA transactions (i.e. global transactions) at all, as they slow processing dramatically.