Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Flink: Watermarking with Late Elements

I am doing real-time streaming in Flink where the Kafka is the message queue. I am applying EventTimeSlidingWindow of 120 sec. and slide of 1 sec. I am also inserting the watermark at each second of Event Time.

My concern is what happened if the element will come late, after the watermark? Now I my case, Flink simply discard the message which come after its respective watermark. Is there any mechanism provided by the filnk to handle such late message, like maintaining separate window? I have also gone through the documentation but I did not get clear about it.

like image 295
kadsank Avatar asked Oct 26 '25 08:10

kadsank


1 Answers

Apache Flink has a concept called allowed lateness for the windows to handle data that arrives after a watermark.

like image 111
Robert Metzger Avatar answered Oct 28 '25 02:10

Robert Metzger



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!