I am looking for advise and best practice regarding field name case sensitivity in ElasticSearch and whether there is a global configuration to make field names case insensitive. Also, if it is possible to disable ES from adding different fields if it does not exists in the mapping.
here is an example to illustrate the point;
1- create mapping with one field "name" in lowercase
curl -XPUT http://localhost:9200/twitter/user/_mapping -d '{
"user" : {
"properties" : {
"name" : { "type" : "string" }
}
}
}'
2- Index a document, using a different case for the name field (NAME)
curl -POST http://localhost:9200/twitter/user/1 -d '{
"NAME" : "Yasir"
}'
In the Elasticsearch logs, I noticed that the mapping is updated.
[2014-01-26 20:58:19,074][INFO ][cluster.metadata ] [Mad-Dog] [twitter] update_mapping [user] (dynamic)
3- check the mapping, you will notice a new field has been added "NAME"
curl -XGET http://localhost:9200/twitter/user/_mapping?pretty
{
"user" : {
"properties" : {
"NAME" : {
"type" : "string"
},
"name" : {
"type" : "string"
}
}
}
}
Thanks Yasir
This would also help out by preventing field explosions and make querying and using the elastic api easier. Databases have a variety of sensitivities. SQL, by default, is case insensitive to identifiers and keywords, but case sensitive to data. JSON is case sensitive to both field names and data.
Elasticsearch ECS Data Typesit is case sensitive, requires use of regex (which worth mentioning is not the full PCRE spec) aggregation max character length.
The keyword family includes the following field types: keyword , which is used for structured content such as IDs, email addresses, hostnames, status codes, zip codes, or tags. constant_keyword for keyword fields that always contain the same value. wildcard for unstructured machine-generated content.
You can disbale the automatic creation of the mapping in the configs, like mentioned in the documentation.
Just set action.auto_create_index
to false
.
The field names can not be case-insensitive, since the naming belongs to you. I would suggest to only use lowercase for that. However, you can search in your values case-insensitive.
EDIT:
Like @javanna's comment says, this does not disable the dynamic mapping.
Therefore, you have to set index.mapper.dynamic
to false
.
As a result, non declared fields will be ignored.
If you want elasticsearch to throw an error instead, you have to set it to strict
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With