I'm trying to create a custom coder AvroDeterministicCoder which is based on AvroCoder but returns true for isDeterministic.
In my example Dataflow I register this coder for all my PCollections. However, when I run my test it ends up throwing an exception:
the key coder must be deterministic for grouping.
Using my debugger, I can tell that its using the coder AvroCoder.
In AvroDeterministicCoder do I need to add code to cause AvroDeterministicCoder to be deserialized as AvroDeterministicCoder and not as an AvroCoder?
It was a bug in my code; I needed to override the various of methods to return type AvroDeterministicCoder.
Here's the working code: AvroDeterministicCoder.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With