Kafka transactions on multiple clusters

Question

I could not find the answer in the transaction API. I think it's probably not possible looking at the design under the hood but I wanted a confirmation: is it possible to have a Kafka transaction if the consumer and the producer are connected to different Kafka clusters? Meaning the topics I am consuming from and the topics I am pushing to are on 2 separate Kafka clusters.

Answer 1

You are correct, a Kafka transaction cannot span across multiple clusters.

Both the consumer and producer must use the same cluster. The main reason is that both the offsets from the consumer and the records from the producer have to be committed together in the transaction.