Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to create a custom coder?

I'm trying to create a custom coder AvroDeterministicCoder which is based on AvroCoder but returns true for isDeterministic.

In my example Dataflow I register this coder for all my PCollections. However, when I run my test it ends up throwing an exception: the key coder must be deterministic for grouping.

Using my debugger, I can tell that its using the coder AvroCoder.

In AvroDeterministicCoder do I need to add code to cause AvroDeterministicCoder to be deserialized as AvroDeterministicCoder and not as an AvroCoder?

like image 309
Jeremy Lewi Avatar asked Jul 27 '26 22:07

Jeremy Lewi


1 Answers

It was a bug in my code; I needed to override the various of methods to return type AvroDeterministicCoder.

Here's the working code: AvroDeterministicCoder.

like image 68
Jeremy Lewi Avatar answered Jul 30 '26 02:07

Jeremy Lewi



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!