I have a simple pipeline in dataflow 2.1 sdk. Which reads data from pubsub then applies a DoFn to it.
PCollection<MyClass> e = streamData.apply("ToE", ParDo.of(new MyDoFNClass()));
Getting below error on this pipeline:
java.lang.IllegalStateException: Unable to return a default Coder for ToEvents/ParMultiDo(MyDoFNClass).out0 [PCollection]. Correct one of the following root causes: No Coder has been manually specified; you may do so using .setCoder(). Inferring a Coder from the CoderRegistry failed: Unable to provide a Coder for com.X.X.model.MyClass.
MyDoFn class is below:
@DefaultCoder(AvroCoder.class)
public class MyClass{
public long id;
public HashMap<String,HashSet<String>> a;
@SerializedName("a")
public Integer Id;
@SerializedName("ae")
public String ae;
}
Found the solution just neeeded to add implements Serializable
to MyClass
@DefaultCoder(AvroCoder.class)
public class MyClass implements Serializable {
public long id;
public HashMap<String,HashSet<String>> a;
@SerializedName("a")
public Integer Id;
@SerializedName("ae")
public String ae;
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With