How to serialize java.util.Properties to and from JSON file?

Question

I have variable of type java.util.Properties. I am trying to write it to a JSON file, and as well as read from that file.

The Properties variable looks something like below:

Properties inner3 = new Properties();
inner3.put("i1", 1);
inner3.put("i2", 100);

Properties inner2 = new Properties();
inner2.put("aStringProp", "aStringValue");
inner2.put("inner3", inner3);

Properties inner1 = new Properties();
inner1.put("aBoolProp", true);
inner1.put("inner2", inner2);

Properties topLevelProp = new Properties();
topLevelProp.put("count", 1000000);
topLevelProp.put("size", 1);
topLevelProp.put("inner1", inner1);

Naturally, when I serialize the topLevelProp to JSON I expect the result to be as below.

{
  "inner1": {
    "inner2": {
      "aStringProp": "aStringValue",
      "inner3": {
        "i2": 100,
        "i1": 1
      }
    },
    "aBoolProp": true
  },
  "size": 1,
  "count": 1000000
}

The above JSON result can be produced by using Gson in a pretty straight forward way, but when it is fed the same JSON string to desrialize, it fails.

Gson gson = new GsonBuilder().create();
String json = gson.toJson(topLevelProp); //{"inner1":{"inner2":{"aStringProp":"aStringValue","inner3":{"i2":100,"i1":1}},"aBoolProp":true},"size":1,"count":1000000}

//following line throws error: Expected a string but was BEGIN_OBJECT at line 1 column 12 path $.
Properties propObj = gson.fromJson(json, Properties.class); 

Tried with Jackson as well:

ObjectMapper mapper = new ObjectMapper();
mapper.configure(MapperFeature.PROPAGATE_TRANSIENT_MARKER, true);
mapper.setVisibility(PropertyAccessor.ALL, Visibility.NONE);
mapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY);
File file = new File("configs/config1.json");
mapper.writeValue(file, topLevelProp);

The last line throws error:

com.fasterxml.jackson.databind.JsonMappingException: java.util.Properties cannot be cast to java.lang.String (through reference chain: java.util.Properties["inner1"])

Tried to desrialize from the string as follows and it failed with the following error:

Properties jckProp = JsonSerializer.mapper.readValue(json, Properties.class);

Can not deserialize instance of java.lang.String out of START_OBJECT token at [Source: {"inner1":{"inner2":{"aStringProp":"aStringValue","inner3":{"i2":100,"i1":1}},"aBoolProp":true},"size":1,"count":1000000}; line: 1, column: 11] (through reference chain: java.util.Properties["inner1"])

How this can be handled?

Update: Following the idea of cricket_007, found com.fasterxml.jackson.databind.node.ObjectNode, can be used as follows:

ObjectNode jckProp = JsonSerializer.mapper.readValue(json, ObjectNode.class);
System.out.println(jckProp.get("size").asInt());
System.out.println("jckProp: " + jckProp);
System.out.println("jckProp.inner: " + jckProp.get("inner1"));

I think this can be the way forward for me, as I mostly have to read from JSON file.

Answer 1

The problem you have is that you are misusing java.util.Properties: it is NOT a multi-level tree structure, but a simple String-to-String map. So while it is technically possibly to add non-String property values (partly since this class was added before Java generics, which made allowed better type safety), this should not be done. For nested structured, use java.util.Map or specific tree data structures.

As to Properties, javadocs say for example:

The Properties class represents a persistent set of properties.
The Properties can be saved to a stream or loaded from a stream.
Each key and its corresponding value in the property list is a string.
...
If the store or save method is called on a "compromised" Properties    
object that contains a non-String key or value, the call will fail. 

Now: if and when you have such "compromised" Properties instance, your best bet with Jackson or Gson is to construct a java.util.Map (or perhaps older Hashtable), and serialize it. That should work without issues.

Answer 2

As it was said above by StaxMan, you're misusing the Properties class and you're close about having heavy issues for using it like that due to lack of type information. However, you might also face the same case for weakly-typed maps. If it's a must for you, then you can use your custom Gson JsonDeserializer (note the JSON arrays issue):

final class PropertiesJsonDeserializer
        implements JsonDeserializer<Properties> {

    private static final JsonDeserializer<Properties> propertiesJsonDeserializer = new PropertiesJsonDeserializer();

    private PropertiesJsonDeserializer() {
    }

    static JsonDeserializer<Properties> getPropertiesJsonDeserializer() {
        return propertiesJsonDeserializer;
    }

    @Override
    public Properties deserialize(final JsonElement jsonElement, final Type type, final JsonDeserializationContext context)
            throws JsonParseException {
        final Properties properties = new Properties();
        final JsonObject jsonObject = jsonElement.getAsJsonObject();
        for ( final Entry<String, JsonElement> e : jsonObject.entrySet() ) {
            properties.put(e.getKey(), parseValue(context, e.getValue()));
        }
        return properties;
    }

    private static Object parseValue(final JsonDeserializationContext context, final JsonElement valueElement) {
        if ( valueElement instanceof JsonObject ) {
            return context.deserialize(valueElement, Properties.class);
        }
        if ( valueElement instanceof JsonPrimitive ) {
            final JsonPrimitive valuePrimitive = valueElement.getAsJsonPrimitive();
            if ( valuePrimitive.isBoolean() ) {
                return context.deserialize(valueElement, Boolean.class);
            }
            if ( valuePrimitive.isNumber() ) {
                return context.deserialize(valueElement, Number.class); // depends on the JSON literal due to the lack of real number type info
            }
            if ( valuePrimitive.isString() ) {
                return context.deserialize(valueElement, String.class);
            }
            throw new AssertionError();
        }
        if ( valueElement instanceof JsonArray ) {
            throw new UnsupportedOperationException("Arrays are unsupported due to lack of type information (a generic list or a concrete type array?)");
        }
        if ( valueElement instanceof JsonNull ) {
            throw new UnsupportedOperationException("Nulls cannot be deserialized");
        }
        throw new AssertionError("Must never happen");
    }

}

Hence, it might be used like this:

private static final Gson gson = new GsonBuilder()
        .registerTypeAdapter(Properties.class, getPropertiesJsonDeserializer())
        .create();

public static void main(final String... args) {
    final Properties outgoingProperties = createProperties();
    out.println(outgoingProperties);
    final String json = gson.toJson(outgoingProperties);
    out.println(json);
    final Properties incomingProperties = gson.fromJson(json, Properties.class);
    out.println(incomingProperties);
}

private static Properties createProperties() {
    final Properties inner3 = new Properties();
    inner3.put("i1", 1);
    inner3.put("i2", 100);
    final Properties inner2 = new Properties();
    inner2.put("aStringProp", "aStringValue");
    inner2.put("inner3", inner3);
    final Properties inner1 = new Properties();
    inner1.put("aBoolProp", true);
    inner1.put("inner2", inner2);
    final Properties topLevelProp = new Properties();
    topLevelProp.put("count", 1000000);
    topLevelProp.put("size", 1);
    topLevelProp.put("inner1", inner1);
    return topLevelProp;
}

with the following output:

{inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000}
{"inner1":{"inner2":{"aStringProp":"aStringValue","inner3": {"i2":100,"i1":1}},"aBoolProp":true},"size":1,"count":1000000}
{inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000}

---

Type info injection

You could save some type information though, if you inject the type information in the result JSON. Let's assume you are fine with storing numeric values as not primitives, but JSON objects having two keys like _$T and _$V to hold the actual type (a class indeed, not any java.reflect.Type, unfortunately) and the associated value respectively in order to restore the real type of the property. This can be applied to arrays either, but it's still not possible to hold a parameterized type due to the lack of type paremerization for instances that are parameterized somehow (unless you can reach it via a Class instance):

final class PropertiesJsonDeserializer
        implements JsonDeserializer<Properties> {

    private static final JsonDeserializer<Properties> propertiesJsonDeserializer = new PropertiesJsonDeserializer();

    private PropertiesJsonDeserializer() {
    }

    static JsonDeserializer<Properties> getPropertiesJsonDeserializer() {
        return propertiesJsonDeserializer;
    }

    @Override
    public Properties deserialize(final JsonElement jsonElement, final Type type, final JsonDeserializationContext context)
            throws JsonParseException {
        final Properties properties = new Properties();
        final JsonObject jsonObject = jsonElement.getAsJsonObject();
        for ( final Entry<String, JsonElement> e : jsonObject.entrySet() ) {
            properties.put(e.getKey(), parseValue(context, e.getValue()));
        }
        return properties;
    }

    private static Object parseValue(final JsonDeserializationContext context, final JsonElement valueElement) {
        if ( valueElement instanceof JsonObject ) {
            return context.deserialize(valueElement, Properties.class);
        }
        if ( valueElement instanceof JsonPrimitive ) {
            final JsonPrimitive valuePrimitive = valueElement.getAsJsonPrimitive();
            if ( valuePrimitive.isBoolean() ) {
                return context.deserialize(valueElement, Boolean.class);
            }
            if ( valuePrimitive.isNumber() ) {
                return context.deserialize(valueElement, Number.class); // depends on the JSON literal due to the lack of real number type info
            }
            if ( valuePrimitive.isString() ) {
                return context.deserialize(valueElement, String.class);
            }
            throw new AssertionError();
        }
        if ( valueElement instanceof JsonArray ) {
            throw new UnsupportedOperationException("Arrays are unsupported due to lack of type information (a generic list or a concrete type array?)");
        }
        if ( valueElement instanceof JsonNull ) {
            throw new UnsupportedOperationException("Nulls cannot be deserialized");
        }
        throw new AssertionError("Must never happen");
    }

}

final class TypeAwarePropertiesSerializer
        implements JsonSerializer<Properties> {

    private static final JsonSerializer<Properties> typeAwarePropertiesSerializer = new TypeAwarePropertiesSerializer();

    private TypeAwarePropertiesSerializer() {
    }

    static JsonSerializer<Properties> getTypeAwarePropertiesSerializer() {
        return typeAwarePropertiesSerializer;
    }

    @Override
    public JsonElement serialize(final Properties properties, final Type type, final JsonSerializationContext context) {
        final JsonObject propertiesJson = new JsonObject();
        for ( final Entry<Object, Object> entry : properties.entrySet() ) {
            final String property = (String) entry.getKey();
            final Object value = entry.getValue();
            if ( value instanceof Boolean ) {
                propertiesJson.addProperty(property, (Boolean) value);
            } else if ( value instanceof Character ) {
                propertiesJson.addProperty(property, (Character) value);
            } else if ( value instanceof Number ) {
                final JsonObject wrapperJson = newWrapperJson(value);
                wrapperJson.addProperty("_$V", (Number) value);
                propertiesJson.add(property, wrapperJson);
            } else if ( value instanceof String ) {
                propertiesJson.addProperty(property, (String) value);
            } else if ( value instanceof Properties || value instanceof Collection || value instanceof Map ) {
                propertiesJson.add(property, context.serialize(value));
            } else if ( value != null ) {
                final Class<?> aClass = value.getClass();
                if ( aClass.isArray() ) {
                    final JsonObject wrapperJson = newWrapperJson(value);
                    wrapperJson.add("_$V", context.serialize(value));
                    propertiesJson.add(property, wrapperJson);
                } else {
                    throw new UnsupportedOperationException("Cannot process: " + value);
                }
            } else /* now the value is always null, Properties cannot hold nulls */ {
                throw new AssertionError("Must never happen");
            }
        }
        return propertiesJson;
    }

    private static JsonObject newWrapperJson(final Object value) {
        final JsonObject wrapperJson = new JsonObject();
        wrapperJson.addProperty("_$T", value.getClass().getName());
        return wrapperJson;
    }

}

final class TypeAwarePropertiesDeserializer
        implements JsonDeserializer<Properties> {

    private static final JsonDeserializer<Properties> typeAwarePropertiesDeserializer = new TypeAwarePropertiesDeserializer();

    private TypeAwarePropertiesDeserializer() {
    }

    static JsonDeserializer<Properties> getTypeAwarePropertiesDeserializer() {
        return typeAwarePropertiesDeserializer;
    }

    @Override
    public Properties deserialize(final JsonElement jsonElement, final Type type, final JsonDeserializationContext context)
            throws JsonParseException {
        try {
            final Properties properties = new Properties();
            final JsonObject jsonObject = jsonElement.getAsJsonObject();
            for ( final Entry<String, JsonElement> e : jsonObject.entrySet() ) {
                properties.put(e.getKey(), parseValue(context, e.getValue()));
            }
            return properties;
        } catch ( final ClassNotFoundException ex ) {
            throw new JsonParseException(ex);
        }
    }

    private static Object parseValue(final JsonDeserializationContext context, final JsonElement valueElement)
            throws ClassNotFoundException {
        if ( valueElement instanceof JsonObject ) {
            final JsonObject valueObject = valueElement.getAsJsonObject();
            if ( isWrapperJson(valueObject) ) {
                return context.deserialize(getWrapperValueObject(valueObject), getWrapperClass(valueObject));
            }
            return context.deserialize(valueElement, Properties.class);
        }
        if ( valueElement instanceof JsonPrimitive ) {
            final JsonPrimitive valuePrimitive = valueElement.getAsJsonPrimitive();
            if ( valuePrimitive.isBoolean() ) {
                return context.deserialize(valueElement, Boolean.class);
            }
            if ( valuePrimitive.isNumber() ) {
                throw new AssertionError("Must never happen because of 'unboxing' above");
            }
            if ( valuePrimitive.isString() ) {
                return context.deserialize(valueElement, String.class);
            }
            throw new AssertionError("Must never happen");
        }
        if ( valueElement instanceof JsonArray ) {
            return context.deserialize(valueElement, Collection.class);
        }
        if ( valueElement instanceof JsonNull ) {
            throw new UnsupportedOperationException("Nulls cannot be deserialized");
        }
        throw new AssertionError("Must never happen");
    }

    private static boolean isWrapperJson(final JsonObject valueObject) {
        return valueObject.has("_$T") && valueObject.has("_$V");
    }

    private static Class<?> getWrapperClass(final JsonObject valueObject)
            throws ClassNotFoundException {
        return Class.forName(valueObject.get("_$T").getAsJsonPrimitive().getAsString());
    }

    private static JsonElement getWrapperValueObject(final JsonObject valueObject) {
        return valueObject.get("_$V");
    }

}

Now the topLevelProp can be filled also with:

topLevelProp.put("ARRAY", new String[]{ "foo", "bar" });
topLevelProp.put("RAW_LIST", asList("foo", "bar"));

if you have these special JSON deserializers applied:

private static final Gson typeAwareGson = new GsonBuilder()
        .registerTypeAdapter(Properties.class, getTypeAwarePropertiesSerializer())
        .registerTypeAdapter(Properties.class, getTypeAwarePropertiesDeserializer())
        .create();

A sample output:

{RAW_LIST=[foo, bar], inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000, ARRAY=[Ljava.lang.String;@b81eda8}
{"RAW_LIST":["foo","bar"],"inner1":{"inner2":{"aStringProp":"aStringValue","inner3":{"i2":{"_$T":"java.lang.Integer","_$V":100},"i1":{"_$T":"java.lang.Integer","_$V":1}}},"aBoolProp":true},"size":{"_$T":"java.lang.Integer","_$V":1},"count":{"_$T":"java.lang.Integer","_$V":1000000},"ARRAY":{"_$T":"[Ljava.lang.String;","_$V":["foo","bar"]}}
{RAW_LIST=[foo, bar], inner1={inner2={aStringProp=aStringValue, inner3={i2=100, i1=1}}, aBoolProp=true}, size=1, count=1000000, ARRAY=[Ljava.lang.String;@e2144e4}

Summarizing up two approaches, you might want to eliminate the need of weak-typing and introduce explicit POJO mappings if possible.