How to pass through properties in JSON messages with Jackson and MongoDB?

Question

We have a microservice which gets some JSON data from the queue, processes it a little bit and sends the result of processing further on - again via queue. In the microservice we don't work with JSONObject an likes directly, we map JSON onto Java classes using Jackson.

When processing, the microservice is only interested in a some properties of the incoming message, not all of them. Imagine it just receives

{
    "operand1": 3,
    "operand2": 5,
    /* other properties may come here */
}

And sends:

{
    "operand1": 3,
    "operand2": 5,
    "multiplicationResult": 15,
    /* other properties may come here */
}

How can I tunnell or pass-through other properties of the message which I'm not interested in this service without explicitly mapping them in my classes?

For the purposes of this microservice it would be enough to have a structure like:

public class Task {
   public double operand1;
   public double operand2;
   public double multiplicationResult;
}

However if I don't map all of the other properties, they will be lost.

If I do map them then I'll have to update the model of this microservice every time the structure of the message changes which takes effort and is error-prone.

Answer 1

The simplest way is to use Map instead of custom POJO in case of agile structure:

It is easy to read from JSON, e.g. using JsonParser parser (java docs here):

Map<String, Object> fields =
      parser.readValueAs(new TypeReference<Map<String, Object>>() {});

It is easy to write into MongoDB using BasicDBObject (java docs here):

DBCollection collection = db.getCollection("tasks");
collection.insert(new BasicDBObject(fields));

You can even achieve it by wrapping Map with Task like this:

public class Task {
    private final Map<String, Object> fields;

    public final double operand1;
    // And so on...

    @JsonCreator
    public Task(Map<String, Object> fields) {
        this.fields = fields;

        this.operand1 = 0; /* Extract desired values from the Map */
    }

    @JsonValue
    public Map<String, Object> toMap() {
        return this.fields;
    }
}

---

It is also possible to use custom JsonDeserializer if it is required (Task must be annotated with @JsonDeserialize(using = TaskDeserializer.class) in that case):

public class TaskDeserializer extends JsonDeserializer<Task> {
    @Override
    public Task deserialize(JsonParser parser, DeserializationContext context)
            throws IOException, JsonProcessingException {
        Map<String, Object> fields = parser.readValueAs(new TypeReference<Map<String, Object>>() {});
        return new Task(fields);
}

Answer 2

As it was said earlier, @JsonAnyGetter and @JsonAnySetter can be the best choice for you. I think that you can do it as much flexible as much type-safe it can be.

The very first thing that comes to my mind is accurate separating the properties you need and the whole rest.

The core

Calculation.java

A simple immutable "calculation" object. Of course, it can be designed in any other way, but immutability makes it simpler and more reliable, I believe.

final class Calculation {

    private final double a;
    private final double b;
    private final Operation operation;
    private final Double result;

    private Calculation(final double a, final double b, final Operation operation, final Double result) {
        this.a = a;
        this.b = b;
        this.operation = operation;
        this.result = result;
    }

    static Calculation calculation(final double a, final double b, final Operation operation, final Double result) {
        return new Calculation(a, b, operation, result);
    }

    Calculation calculate() {
        return new Calculation(a, b, operation, operation.applyAsDouble(a, b));
    }

    double getA() {
        return a;
    }

    double getB() {
        return b;
    }

    Operation getOperation() {
        return operation;
    }

    Double getResult() {
        return result;
    }

}

Operation.java

A simple calculation strategy defined in an enumeration since Jackson works with enumerations really good.

enum Operation
        implements DoubleBinaryOperator {

    ADD {
        @Override
        public double applyAsDouble(final double a, final double b) {
            return a + b;
        }
    },

    SUBTRACT {
        @Override
        public double applyAsDouble(final double a, final double b) {
            return a - b;
        }
    },

    MULTIPLY {
        @Override
        public double applyAsDouble(final double a, final double b) {
            return a * b;
        }
    },

    DIVIDE {
        @Override
        public double applyAsDouble(final double a, final double b) {
            return a / b;
        }
    }

}

Jackson mappings

AbstractTask.java

Note that this class is intended to supply a value, but collect the rest into a satellite map managed by the methods annotated with @JsonAnySetter and @JsonAnyGetter. The map and the methods can be safely declared private since Jackson does not really care the protection level (and this is great). Also, it's designed in immutable manner except of underlying map that can be just shallow-copied to a new value.

abstract class AbstractTask<V>
        implements Supplier<V> {

    @JsonIgnore
    private final Map<String, Object> rest = new LinkedHashMap<>();

    protected abstract AbstractTask<V> toTask(V value);

    final <T extends AbstractTask<V>> T with(final V value) {
        final AbstractTask<V> dto = toTask(value);
        dto.rest.putAll(rest);
        @SuppressWarnings("unchecked")
        final T castDto = (T) dto;
        return castDto;
    }

    @JsonAnySetter
    @SuppressWarnings("unused")
    private void set(final String name, final Object value) {
        rest.put(name, value);
    }

    @JsonAnyGetter
    @SuppressWarnings("unused")
    private Map<String, Object> getRest() {
        return rest;
    }

}

CalculationTask.java

Here is a class the defines a concrete calculation task. Again, Jackson works perfectly with private fields and methods, so the entire complexity can be encapsulated. One disadvantage I can see is that JSON properties are declared both for serializing and deserializing, but it can be also considered an advantage as well. Note that @JsonGetter arguments are not necessary here as such, but I just doubled the property names both for in- and out- operations. No tasks are intended to be instantiated manually - let just Jackson do it.

final class CalculationTask
        extends AbstractTask<Calculation> {

    private final Calculation calculation;

    private CalculationTask(final Calculation calculation) {
        this.calculation = calculation;
    }

    @JsonCreator
    @SuppressWarnings("unused")
    private static CalculationTask calculationTask(
            @JsonProperty("a") final double a,
            @JsonProperty("b") final double b,
            @JsonProperty("operation") final Operation operation,
            @JsonProperty("result") final Double result
    ) {
        return new CalculationTask(calculation(a, b, operation, result));
    }

    @Override
    public Calculation get() {
        return calculation;
    }

    @Override
    protected AbstractTask<Calculation> toTask(final Calculation calculation) {
        return new CalculationTask(calculation);
    }

    @JsonGetter("a")
    @SuppressWarnings("unused")
    private double getA() {
        return calculation.getA();
    }

    @JsonGetter("b")
    @SuppressWarnings("unused")
    private double getB() {
        return calculation.getB();
    }

    @JsonGetter("operation")
    @SuppressWarnings("unused")
    private Operation getOperation() {
        return calculation.getOperation();
    }

    @JsonGetter("result")
    @SuppressWarnings("unused")
    private Double getResult() {
        return calculation.getResult();
    }

}

Client-server interaction

CalculationController.java

Here is a simple GET/PUT/DELETE controller for integration testing, or just to be manually tested with curl.

@RestController
@RequestMapping("/")
final class CalculationController {

    private final CalculationService processService;

    @Autowired
    @SuppressWarnings("unused")
    CalculationController(final CalculationService processService) {
        this.processService = processService;
    }

    @RequestMapping(method = GET, value = "{id}")
    @ResponseStatus(OK)
    @SuppressWarnings("unused")
    CalculationTask get(@PathVariable("id") final String id) {
        return processService.get(id);
    }

    @RequestMapping(method = PUT, value = "{id}")
    @ResponseStatus(NO_CONTENT)
    @SuppressWarnings("unused")
    void put(@PathVariable("id") final String id, @RequestBody final CalculationTask task) {
        processService.put(id, task);
    }

    @RequestMapping(method = DELETE, value = "{id}")
    @ResponseStatus(NO_CONTENT)
    @SuppressWarnings("unused")
    void delete(@PathVariable("id") final String id) {
        processService.delete(id);
    }

}

ControllerExceptionHandler.java

Since the get and delete methods declared below in the DAO class throw NoSuchElementException, the exception can be easily mapped to HTTP 404.

@ControllerAdvice
final class ControllerExceptionHandler {

    @ResponseStatus(NOT_FOUND)
    @ExceptionHandler(NoSuchElementException.class)
    @SuppressWarnings("unused")
    void handleNotFound() {
    }

}

The application itself

CalculationService.java

Just a simple service that contains some "business" logic.

@Service
final class CalculationService {

    private final CalculationDao processDao;

    @Autowired
    CalculationService(final CalculationDao processDao) {
        this.processDao = processDao;
    }

    CalculationTask get(final String id) {
        return processDao.get(id);
    }

    void put(final String id, final CalculationTask task) {
        processDao.put(id, task.with(task.get().calculate()));
    }

    void delete(final String id) {
        processDao.delete(id);
    }

}

The data layer

CalculationMapping.java

Just a holder class in order to work with MongoDB repositories in Spring Data specifying the target MongoDB document collection name.

@Document(collection = "calculations")
public final class CalculationTaskMapping
        extends org.bson.Document {

    @Id
    @SuppressWarnings("unused")
    private String id;

}

ICalculationRepository.java

A Spring Data MongoDB CRUD repository for the CalculationMapping class. This repository is used below.

@Repository
interface ICalculationRepository
        extends MongoRepository<CalculationTaskMapping, String> {
}

CalculationDao.java

The DAO component does not make much work in the demo itself, and it's more about delegating the persistence job to its super class and being easy to find by Spring Framework.

@Component
final class CalculationDao
        extends AbstractDao<CalculationTask, CalculationTaskMapping, String> {

    @Autowired
    CalculationDao(@SuppressWarnings("TypeMayBeWeakened") final ICalculationRepository calculationRepository, final ObjectMapper objectMapper) {
        super(CalculationTaskMapping.class, calculationRepository, objectMapper);
    }

}

AbstractDao.java

This is the heart of persisting the whole original object. The ObjectMapper instance is used to convert tasks to their respective task mappings (see the convertValue method) according to the serialization rules specified with the Jackson annotations. Since the demo uses Spring Data MongoDB, the mapping classes are effectively Map<String, Object> and inherit the Document class. Unfortunately, Map-oriented mappings do not seem to work with Spring Data MongoDB annotations like @Id, @Field, etc (see more at How do I combine java.util.Map-based mappings with the Spring Data MongoDB annotations (@Id, @Field, ...)?). However, it can be justified as long as you do not want to map arbitrary documents.

abstract class AbstractDao<T, M extends Document, ID extends Serializable> {

    private final Class<M> mappingClass;
    private final CrudRepository<M, ID> crudRepository;
    private final ObjectMapper objectMapper;

    protected AbstractDao(final Class<M> mappingClass, final CrudRepository<M, ID> crudRepository, final ObjectMapper objectMapper) {
        this.mappingClass = mappingClass;
        this.crudRepository = crudRepository;
        this.objectMapper = objectMapper;
    }

    final void put(final ID id, final T task) {
        final M taskMapping = objectMapper.convertValue(task, mappingClass);
        taskMapping.put(ID_FIELD_NAME, id);
        if ( crudRepository.exists(id) ) {
            crudRepository.delete(id);
        }
        crudRepository.save(taskMapping);
    }

    final CalculationTask get(final ID id) {
        final Map<String, Object> rawTask = crudRepository.findOne(id);
        if ( rawTask == null ) {
            throw new NoSuchElementException();
        }
        rawTask.remove(ID_FIELD_NAME);
        return objectMapper.convertValue(rawTask, CalculationTask.class);
    }

    final void delete(final ID id) {
        final M taskMapping = crudRepository.findOne(id);
        if ( taskMapping == null ) {
            throw new NoSuchElementException();
        }
        crudRepository.delete(id);
    }

}

The Spring Boot application

EntryPoint.class

And a Spring Boot demo that runs all of it as a single HTTP application listening to port 9000.

@SpringBootApplication
@Configuration
@EnableWebMvc
public class EntryPoint
        extends SpringBootServletInitializer {

    @Override
    protected SpringApplicationBuilder configure(final SpringApplicationBuilder builder) {
        return builder.sources(EntryPoint.class);
    }

    @Bean
    EmbeddedServletContainerCustomizer embeddedServletContainerCustomizer() {
        return c -> c.setPort(9000);
    }

    @Bean
    ObjectMapper objectMapper() {
        return new ObjectMapper()
                .setSerializationInclusion(NON_NULL)
                .configure(FAIL_ON_UNKNOWN_PROPERTIES, false);
    }

    @SuppressWarnings("resource")
    public static void main(final String... args) {
        SpringApplication.run(EntryPoint.class, args);
    }

}

Testing the application with curl

(mongodb-shell)
> use test

switched to db local

(bash)
$ curl -v -X GET http://localhost:9000/foo

> GET /foo HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:9000
> Accept: /
>
< HTTP/1.1 404
< Content-Length: 0
< Date: Thu, 15 Dec 2016 10:07:40 GMT
<

(mongodb-shell)
> db.calculations.find()

(empty)

(bash)
$ curl -v -X PUT -H 'Content-Type: application/json' \
    --data '{"a":3,"b":4,"operation":"MULTIPLY","result":12,"foo":"FOO","bar":"BAR"}' \
    http://localhost:9000/foo

> PUT /foo HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:9000
> Accept: /
> Content-Type: application/json
> Content-Length: 72
>
< HTTP/1.1 204
< Date: Thu, 15 Dec 2016 10:11:13 GMT
<

(mongodb-shell)
> db.calculations.find()

{ "_id" : "foo", "_class" : "q41036545.CalculationTaskMapping", "a" : 3, "b" : 4, "operation" : "MULTIPLY", "result" : 12, "foo" : "FOO", "bar" : "BAR" }

(bash)
$ curl -v -X GET http://localhost:9000/foo

> GET /foo HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:9000
> Accept: /
>
< HTTP/1.1 200
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Thu, 15 Dec 2016 10:16:33 GMT
<
{"a":3.0,"b":4.0,"operation":"MULTIPLY","result":12.0,"foo":"FOO","bar":"BAR"}

(bash)
curl -v -X DELETE http://localhost:9000/foo

> DELETE /foo HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:9000
> Accept: /
>
< HTTP/1.1 204
< Date: Thu, 15 Dec 2016 11:51:26 GMT
<

(mongodb-shell)
> db.calculations.find()

(empty)

The source code can be found at https://github.com/lyubomyr-shaydariv/q41036545