When I use session window with 1h gap and after processing million of messages I get error in logs, probably just for some rows:
TypeError: Cannot convert GlobalWindow to apache_beam.utils.windowed_value._IntervalWindowBase
Code:
grouped_tis = tracking_informations | beam.WindowInto(window.Sessions(session_window_gap)) | beam.GroupByKey() | beam.ParDo(MergeTI()) | "TI model -> json" >> beam.Map(jsons.dump)
Full stack: https://pastebin.com/pqA5pMay
If anyone is experiencing same problem, I solve this problem insert beam.WindowInto(beam.window.GlobalWindows()) between beam.WindowInto(NONGLOBALWINDOW) | beam.GroupByKey() and other Ptransform that causes problem.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With