How to update the mapping in Elasticsearch to change the field datatype and change the type of analyzers in string

Question

While trying to update the mapping I get the following error:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"mapper [timestamp] of different type, current_type [string], merged_type [date]"}],"type":"illegal_argument_exception","reason":"
mapper [timestamp] of different type, current_type [string], merged_type [date]"},"status":400}

I m trying to run the following command on windows

   curl -XPUT localhost:9200/logstash-*/_mapping/log?update_all_types -d "{
    "properties":
    {
        "timestamp": 
        {
            "type": "date", 
            "format": "MM-dd-yyyy HH:mm:ss",
            "fielddata":{"loading" : "lazy"} }
        }
    }";

How I can change the datatype of date field from string to date type with a particular format.

I tried to change the mapping of a string datatype to change it to eager loading and not_analyzed from analyzed, but it gives the following error:

{"root_cause":[{"type":"illegal_argument_exception","reason":"Mapper for [AppName] conflicts with existing mapping in other types:\n[mapper [AppName] has different [index] values, mapper [App
 different [doc_values] values, cannot change from disabled to enabled, mapper [AppName] has different [analyzer]]"}],"type":"illegal_argument_exception","reason":"Mapper for [AppName] conflict with
existing mapping in other types:\n[mapper [AppName] has different [index] values, mapper [AppName] has different [doc_values] values, cannot change from disabled to enabled, mapper [AppName]
rent [analyzer]]"},"status":400}

Here is my query for the same:

 curl -XPUT localhost:9200/logstash-*/_mapping/log?update_all_types -d "{
"properties":
    {"AppName":
        {
        "type": "string", 
        "index" : "not_analyzed",
        "fielddata":{"loading" : "eager"}
        }
    }
}"

However, if I change it from not_analyzed to analyzed it gives a acknowledged=true message. How can I change the analyzer.

Answer 1

You cannot change existing data types mapping. As Elastic docs say:

Although you can add to an existing mapping, you can’t change existing field mappings. If a mapping already exists for a field, data from that field has probably been indexed. If you were to change the field mapping, the indexed data would be wrong and would not be properly searchable.

We can update a mapping to add a new field, but we can’t change an existing field from analyzed to not_analyzed.

Your only option is to create a new index with the new mapping and reindex the data from the old index to the new one.

Answer 2

No, you cannot change a single field definition.

If you want to change the field definition for a single field in a single type, you have little option but to reindex all of the documents in your index.

---

Why can't you change mappings? This article Changing Mapping with Zero Downtime explains,

In order to make your data searchable, your database needs to know what type of data each field contains and how it should be indexed.

If you switch a field type from e.g. a string to a date, all of the data for that field that you already have indexed becomes useless. One way or another, you need to reindex that field.

This applies not just to Elasticsearch, but to any database that uses indices for searching. And if it isn't using indices then it is sacrificing speed for flexibility.

---

What happens when you index a document with an incorrect field type?

A conversion will be attempted. If no valid conversion exists, an exception is thrown.

Elasticsearch: The Definitive Guide has a note regarding an example, a string is entered but long is expected. A conversion will be attempted. But an exception will still be thrown if no valid conversion exists.

[...] if the field is already mapped as type long, then ES will try to convert the string into a long, and throw an exception if it can’t.

---

Can I have the document indexed anyway, ignoring the malformed fields?

Yes. ES5 provides a ignore_malformed mapping parameter. Elasticsearch Reference explains that,

Trying to index the wrong datatype into a field throws an exception by default, and rejects the whole document. The ignore_malformed parameter, if set to true, allows the exception to be ignored. The malformed field is not indexed, but other fields in the document are processed normally.