Kafka - difference between Log end offset(LEO) vs High Watermark(HW)

Question

What is the difference between LEO and HW in Replica ( Leader Replica)?

Will they contain the same number? I can understand HW is the last committed message offset.

When LEO will be updated and how?

Answer 1

The high watermark indicated the offset of messages that are fully replicated, while the end-of-log offset might be larger if there are newly appended records to the leader partition which are not replicated yet.

Consumers can only consume messages up to the high watermark.

See this blog post for more details: http://www.confluent.io/blog/hands-free-kafka-replication-a-lesson-in-operational-simplicity/