Is it possible in Kafka to read messages in reverse manner?

Question

Can be created a new consumer group with a consumer which assigned to existing topiс, but somehow set a preference to consume backward: offset will move from the latest message for the moment to the earliest in every partition?

Answer 1

Kafka topics are meant to be consumed sequentually in the order of appearance within the topic partitions.

However, I see two options to solve your issue:

• You can steer the consumer what data it poll from the topic partition like: Have your consumer seek to the latestet offset, then consume it and then seek to the latest offset minus one but read only one offset. Again seek to the previous offset and so on. Although I have never seen it, this should be possible with the consumer.seek and the ConsumerConfiguration max.poll.records.

• You could use any kind of state store and order it descending by the offset for each partition. Then have another consumer reading the state store in the desired order.