Gson deserialization: How to distinguish between fields that are missing and fields that are explicitly set to null?

Question

I'm trying to implement JSON Merge Patch for a Java (JAX-RS) webservice I'm building.

The gist is that partial updates of a record are done by sending a JSON document to the server that only contains the fields that should be changed.

Given this record

{
  "a": "b",
  "c": {
    "d": "e",
    "f": "g"
  }
}

, the following JSON update document

{
  "a":"z",
  "c": {
    "f": null
  }
}

should set a new value for "a" and delete "f" inside "c".

The latter is the problem. I don't know how I can distinguish between an input where f is missing and an input where f is null. Both, as far as I can tell, would be deserialized to null in the target Java Object.

What do?

Answer 1

I acknowledge mlk's answer, but given that I already have (and would nonetheless need) a POJO representation of the JSON object, I feel mapping automatically is still better than looking up manually.

The challenge with that is that, as I said, both missing and explicit null values are set to null in the corresponding POJO that gson.fromJson(...) would populate. (Unlike e.g. R's NULL and NA, Java only has one representation for "not there".)

However, by modelling my data structure using Java 8's Optionals I can do just that: Distinguish between something that is not set, and something that is set to null. Here's what I ended up with:

1) I replaced all fields in my data objects with Optional<T>.

public class BasicObjectOptional {

    private Optional<String> someKey;
    private Optional<Integer> someNumber;
    private Optional<String> mayBeNull;

    public BasicObjectOptional() {
    }

    public BasicObjectOptional(boolean initialize) {
        if (initialize) {
            someKey = Optional.ofNullable("someValue");
            someNumber = Optional.ofNullable(42);
            mayBeNull = Optional.ofNullable(null);
        }
    }

    @Override
    public String toString() {
        return String.format("someKey = %s, someNumber = %s, mayBeNull = %s",
                                            someKey, someNumber, mayBeNull);
    }

}

Or a nested one:

public class ComplexObjectOptional {

    Optional<String> theTitle;  
    Optional<List<Optional<String>>> stringArray;
    Optional<BasicObjectOptional> theObject;

    public ComplexObjectOptional() {
    }

    public ComplexObjectOptional(boolean initialize) {
        if (initialize) {
            theTitle = Optional.ofNullable("Complex Object");   
            stringArray =    Optional.ofNullable(Arrays.asList(Optional.ofNullable("Hello"),Optional.ofNullable("World")));
            theObject = Optional.ofNullable(new BasicObjectOptional(true));
        }
    }

    @Override
    public String toString() {
        return String.format("theTitle = %s, stringArray = %s, theObject = (%s)", theTitle, stringArray, theObject);
    }   
}

2) Implemented a serializer and deserializer based on this useful SO answer.

public class OptionalTypeAdapter<E> extends TypeAdapter<Optional<E>> {

    public static final TypeAdapterFactory FACTORY = new TypeAdapterFactory() {

        //@Override
        public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> type) {
            Class<T> rawType = (Class<T>) type.getRawType();
            if (rawType != Optional.class) {
                return null;
            }
            final ParameterizedType parameterizedType = (ParameterizedType) type.getType();
            final Type actualType = parameterizedType.getActualTypeArguments()[0];
            final TypeAdapter<?> adapter = gson.getAdapter(TypeToken.get(actualType));
            return new OptionalTypeAdapter(adapter);
        }
    };
    private final TypeAdapter<E> adapter;

    public OptionalTypeAdapter(TypeAdapter<E> adapter) {
        this.adapter = adapter;
    }

    @Override
    public void write(JsonWriter out, Optional<E> value) throws IOException {
        if(value == null || !value.isPresent()){
            out.nullValue();
        } else {
            adapter.write(out, value.get());
        }
    }

    @Override
    public Optional<E> read(JsonReader in) throws IOException {
        final JsonToken peek = in.peek();
        if(peek != JsonToken.NULL){
            return Optional.ofNullable(adapter.read(in));
        }
        in.nextNull();
        return Optional.empty();
    }

}

3) Registered this adapter when initializing Gson.

Gson gsonOptFact = new GsonBuilder()
    .serializeNulls() // matter of taste, just for output anyway
    .registerTypeAdapterFactory(OptionalTypeAdapter.FACTORY)
    .create();

This allows me to write JSON such that both null and empty Optional are serialized as null (or simply removed from the output), while at the same time reading JSON into Optional fields such that if the field is null I know it was missing from the JSON input, and if the field is Optional.empty I know it was set to null in the input.

---

Example:

System.out.println(gsonOptFact.toJson(new BasicObjectOptional(true)));
// {"someKey":"someValue","someNumber":42,"mayBeNull":null}

System.out.println(gsonOptFact.toJson(new ComplexObjectOptional(true)));
// {"theTitle":"Complex Object","stringArray":["Hello","World"],"theObject":{"someKey":"someValue","someNumber":42,"mayBeNull":null}}

// Now read back in:
String basic = "{\"someKey\":\"someValue\",\"someNumber\":42,\"mayBeNull\":null}";
String complex = "{\"theTitle\":\"Complex Object\",\"stringArray\":[\"Hello\",\"world\"],\"theObject\":{\"someKey\":\"someValue\",\"someNumber\":42,\"mayBeNull\":null}}";
String complexMissing = "{\"theTitle\":\"Complex Object\",\"theObject\":{\"someKey\":\"someValue\",\"mayBeNull\":null}}";

BasicObjectOptional boo = gsonOptFact.fromJson(basic, BasicObjectOptional.class);
System.out.println(boo);
// someKey = Optional[someValue], someNumber = Optional[42], mayBeNull = Optional.empty

ComplexObjectOptional coo = gsonOptFact.fromJson(complex, ComplexObjectOptional.class);
System.out.println(coo);
// theTitle = Optional[Complex Object], stringArray = Optional[[Optional[Hello], Optional[world]]], theObject = (Optional[someKey = Optional[someValue], someNumber = Optional[42], mayBeNull = Optional.empty])

ComplexObjectOptional coom = gsonOptFact.fromJson(complexMissing, ComplexObjectOptional.class);
System.out.println(coom);
// theTitle = Optional[Complex Object], stringArray = null, theObject = (Optional[someKey = Optional[someValue], someNumber = null, mayBeNull = Optional.empty])

I think this will allow me to integrate JSON Merge Patch with my existing data objects quite well.