Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When to use CoProcess Function in Flink?

I am just trying to understand the use case when to use CoProcessFunction in Flink. Explanation with an example would help me to understand the concept better.

like image 678
Shailendra Avatar asked Mar 07 '26 16:03

Shailendra


1 Answers

A CoProcessFunction is similar to a RichCoFlatMap, but with the addition of also being able to use timers. The timers are useful for expiring state for stale keys, or for raising alarms when keep alive messages fail to arrive, for example.

A CoProcessFunction allows you to use one stream to influence how another is processed, or to enrich another stream. For example, an e-commerce site might have a stream of order events and a stream of shipment events, and they want to create a stream of events for orders that haven't shipped with 24 hours of the order being placed. The two streams can be keyed by the orderId, and connected together. As an order arrives it's recorded in keyed state, and a timer is created to fire 24 hours later. When a shipment event arrives, the state and timer are cleared. If a timer does fire, the state is used to send the order out to the unfilled order service.

For more on this, and examples with code, see connected streams and process function and the labs that accompany those tutorials.

like image 81
David Anderson Avatar answered Mar 09 '26 15:03

David Anderson



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!