Remove elements/objects From Array in ElasticSearch Followed by Matching Query

Question

I'm having issues trying to remove elements/objects from an array in elasticsearch.

This is the mapping for the index:

{
    "example1": {
        "mappings": {
            "doc": {
                "properties": {
                    "locations": {
                        "type": "geo_point"
                    },
                    "postDate": {
                        "type": "date"
                    },
                    "status": {
                        "type": "long"
                    },
                    "user": {
                        "type": "text",
                        "fields": {
                            "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                            }
                        }
                    }
                }
            }
        }
    }
}

And this is an example document.

{
    "_index": "example1",
    "_type": "doc",
    "_id": "8036",
    "_score": 1,
    "_source": {
        "user": "kimchy8036",
        "postDate": "2009-11-15T13:12:00",
        "locations": [
            [
                72.79887719999999,
                21.193036000000003
            ],
            [
                -1.8262150000000001,
                51.178881999999994
            ]
        ]
    }
}

Using the query below, I can add multiple locations.

POST /example1/_update_by_query
{
    "query": {
        "match": {
            "_id": "3"
        }
    },
    "script": {
        "lang": "painless",
        "inline": "ctx._source.locations.add(params.newsupp)",
        "params": {
            "newsupp": [
                -74.00,
                41.12121
            ]
        }
    }
}

But I'm not able to remove array objects from locations. I have tried the query below but it's not working.

POST example1/doc/3/_update
{
    "script": {
        "lang": "painless",
        "inline": "ctx._source.locations.remove(params.tag)",
        "params": {
            "tag": [
                -74.00,
                41.12121
            ]
        }
    }
}

Kindly let me know where i am doing wrong here. I am using elastic version 5.5.2

Answer 1

In painless scripts, Array.remove() method removes by index, not by value.

Here's a working example that removes array elements by value in Elasticsearch script:

POST objects/_update_by_query
{
    "query": {
        ...  // use regular ES query to remove only in relevant documents
    },
    "script": {
        "source": """
            if (ctx._source[params.array_attribute] != null) {
                for (int i=ctx._source[params.array_attribute].length-1; i>=0; i--) {
                    if (ctx._source[params.array_attribute][i] == params.value_to_remove) {
                        ctx._source[params.array_attribute].remove(i);
                    }
                }
            }
        """,
        "params": {
            "array_attribute": "<NAME_OF_ARRAY_PROPERTY_TO_REMOVE_VALUE_IN>",
            "value_to_remove": "<VALUE_TO_REMOVE_FROM_ARRAY>",
        }
    }
}

You might want to simplify script, if your script shall only remove values from one specific array attribute. For example, removing "green" from document's .color_list array:

_doc/001 = {
    "color_list": ["red", "blue", "green"]
}

Script to remove "green":

POST objects/_update_by_query
{
    "query": {
        ...  // use regular ES query to remove only in relevant documents
    },
    "script": {
        "source": """
            for (int i=ctx._source.color_list.length-1; i>=0; i--) {
                if (ctx._source.color_list[i] == params.color_to_remove) {
                    ctx._source.color_list.remove(i);
                }
            }
        """,
        "params": {
            "color_to_remove": "green"
        }
    }
}

Answer 2

Unlike add(), remove() takes the index of the element and remove it.

Your ctx._source.locations in painless is an ArrayList. It has List's remove() method:

E remove(int index)

Removes the element at the specified position in this list (optional operation). ...

See Painless API - List for other methods.

See this answer for example code.