Test Dataflow with DirectRunner and got lots of verifyUnmodifiedThrowingCheckedExceptions

Question

I was testing my Dataflow pipeline using DirectRunner from my Mac and got lots of "WARNING" message like this, may I know how to get rid of them because it is too much that I can not even see my debug message.

Thanks

Apr 05, 2018 2:14:48 PM org.apache.beam.sdk.util.MutationDetectors$CodedValueMutationDetector verifyUnmodifiedThrowingCheckedExceptions
WARNING: Coder of type class org.apache.beam.sdk.coders.SerializableCoder has a #structuralValue method which does not return true when the encoding of the elements is equal. 
Element com.apigee.analytics.platform.core.service.schema.EventRow@4a590d0b

Answer 1

It may help to ensure that all serialized values have proper equals() implementations since SerializableCoder expects them:

The structural value of the object is the object itself. The SerializableCoder should be only used for objects with a proper Object#equals implementation.

You can implement your own Coder for your POJOs. SerializableCoder does not guarantee a deterministic encoding according to docs:

SerializableCoder does not guarantee a deterministic encoding, as Java serialization may produce different binary encodings for two equivalent objects.

This article explains custom coders in details.

Answer 2

I had this same problem. I was using SerializableCoder for a class implementing Serializable, and my tests were failing because the PAssert() containsInAnyOrder() method was not using MyClass.equals() to evaluate object equality. The signature of my equals() method was:

public boolean equals(MyClass other) {...}

All I had to do to fix it was to define equals in terms of Object:

public boolean equals(Object other) {...}

This made the warnings go away, and made the tests pass.

Answer 3

Just add https://projectlombok.org/features/EqualsAndHashCode

@EqualsAndHashCode 
public class YourRecord implements Serializable {