Elasticsearch: Append new elements to nested array of objects

Question

I'm trying to add new items to a nested array of objects from a specific document. I have searched and it seems updates with a partial document doesn't support what I need, it replaces the entire array with the new elements. So I went for a scripted update, which worked as expected via REST API:

PUT /transactions
{
    "mappings": {
        "_doc": {
            "properties": {
                "userId": { "type": "keyword" },
                "transactions": {
                    "type": "object",
                    "properties": {
                        "date": { "type": "date" },
                        "amount": { "type": "double" }
                    }
                }
            }
        }
    }
}

POST /transactions/_doc/1
{
    "userId": "123",
    "transactions": [
        { "date": "2019-07-15T10:32:02Z", "amount": 122 },
        { "date": "2019-07-17T22:09:43Z", "amount": 560 }
    ]
}

POST /transactions/_doc/1/_update
{
    "script": {
        "source": "ctx._source.transactions.addAll(params.transactions)",
        "params": {
            "transactions": [
                { "date": "2019-07-14T21:10:22Z", "amount": 890 },
                { "date": "2019-07-15T15:56:18Z", "amount": 54 }
            ]
        }
    }
}

Then I took the same script to my Java application, this is what the code looks like:

List<Transaction> transactions = Collections.singletonList(new Transaction(320));

Script script = new Script(
    Script.DEFAULT_SCRIPT_TYPE, Script.DEFAULT_SCRIPT_LANG,
    "ctx._source.transactions.addAll(params.transactions);",
    Collections.singletonMap("transactions", transactions));

transportClient
    .prepareUpdate("transactions", "_doc", 1)
    .setFetchSource(false);
    .setScript(script);
    .get();

When the code above is executed, I get the following remote exception:

Caused by: java.io.IOException: can not write type [class com.example.model.Transaction]
    at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:713)
    at org.elasticsearch.common.io.stream.StreamOutput.lambda$static$9(StreamOutput.java:599)
    at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:711)
    at org.elasticsearch.common.io.stream.StreamOutput.lambda$static$11(StreamOutput.java:621)
    at org.elasticsearch.common.io.stream.StreamOutput.writeGenericValue(StreamOutput.java:711)
    at org.elasticsearch.common.io.stream.StreamOutput.writeMap(StreamOutput.java:494)
    at org.elasticsearch.script.Script.writeTo(Script.java:533)
    ...

If I serialize the list into a string I get this exception instead:

org.elasticsearch.common.io.stream.NotSerializableExceptionWrapper: class_cast_exception: Cannot cast java.lang.String to java.util.Collection

Finally, my question is how can I accomplish this with the Java API (Transport Client, Elasticsearch 6.3.2)?

One alternative is to fetch the whole document, deserialize it, append a new transaction and then update the whole document, but it seems overkill and probably degrades the performance.

Answer 1

I just had a curiosity look at the source code, from the writeGenericValue method, it looks like ES doesn't support writes of any custom objects except the ones handled in the multiple if else statements (Object[], List, Map, ReadableInstant, BytesReference).

From the stack trace it looks like script.writeTo is trying to write Transaction objects into the output stream and failing to do so. Converting the Transaction objects to individual Maps and then sending it might solve the problem.