How and why and what for do tombstone records appear in Kafka?

Question

Studying kafka, came across issues deletes tombstone messages ( e.g. while log compaction)

But my question is: how do tombstones even appear there? Who and why wanna use it and for what?

Answer 1

They are written by the producing application (e.g. Producer API, or Kafka Connect source connector) by putting a null in the value part of the message. It denotes a logical deletion for the associated key in the message. If you use log compaction then in time the previous values for that key are actually deleted too.

More info: https://medium.com/@damienthomlutz/deleting-records-in-kafka-aka-tombstones-651114655a16