Is there a way to set a delay for a message sent by a kafka producer?

Question

Or maybe even a way to delay the message received by the consumer. I need to make a function call in nodejs after every 90s, so I want to add a delay of 90s for every kafka message

Answer 1

You could make use of the KafkaProducer configuration linger.ms which is described as:

"[...] adds a small amount of artificial delay — that is, rather than immediately sending out a record the producer will wait for up to the given delay to allow other records to be sent so that the sends can be batched together. [...]"

This configuration defaults to 0 and can be set to any long value.

Depending on the amount of data that your producer sends to a topic, you might also want to increase the configuration batch.size. Otherwise, if this size limit it reached before the delay of linger.ms is over, the KafkaProducer will send the messages before the 90 seconds.

Keep in mind that if you increase linger.ms you may also want to increase delivery.timeout.ms. According to its documentation:

"The value of this config should be greater than or equal to the sum of request.timeout.ms and linger.ms."