Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Disable dynamic mapping creation for only specific indexes on elasticsearch?

I'm trying to disable dynamic mapping creation for only specific indexes, not for all. For some reason I can't put default mapping with 'dynamic' : 'false'. So, here left two options as I can see:

  1. specify property 'index.mapper.dynamic' in file elasticsearch.yml.
  2. put 'index.mapper.dynamic' at index creation time, as described here https://www.elastic.co/guide/en/kibana/current/setup.html#kibana-dynamic-mapping

First option may only accept values: true, false and strict. So there is no way to specify subset of specific indexes (like we do by pattern with property 'action.auto_create_index' https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation).

Second option just not works. I've created index

POST http://localhost:9200/test_idx/
{
    "settings" : {
        "mapper" : {
            "dynamic" : false
        }
    },
    "mappings" : {
        "test_type" : {
            "properties" : {
                "field1" : {
                    "type" : "string"
                }
            }
        }
    }
}

Then checked index settings:

GET http://localhost:9200/test_idx/_settings    
{
    "test_idx" : {
        "settings" : {
            "index" : {
                "mapper" : {
                    "dynamic" : "false"
                },
                "creation_date" : "1445440252221",
                "number_of_shards" : "1",
                "number_of_replicas" : "0",
                "version" : {
                    "created" : "1050299"
                },
                "uuid" : "5QSYSYoORNqCXtdYn51XfA"
            }
        }
    }
}

and mapping:

GET http://localhost:9200/test_idx/_mapping
{
    "test_idx" : {
        "mappings" : {
            "test_type" : {
                "properties" : {
                    "field1" : {
                        "type" : "string"
                    }
                }
            }
        }
    }
}

so far so good, let's index document with undeclared field:

POST http://localhost:9200/test_idx/test_type/1
{
    "field1" : "it's ok, field must be in mapping and in source",
    "somefield" : "but this field must be in source only, not in mapping"
}

Then I've checked mapping again:

GET http://localhost:9200/test_idx/_mapping
{
    "test_idx" : {
        "mappings" : {
            "test_type" : {
                "properties" : {
                    "field1" : {
                        "type" : "string"
                    },
                    "somefield" : {
                        "type" : "string"
                    }
                }
            }
        }
    }
}

As you can see, mapping is extended regardless of index setting "dynamic" : false. I've also tried to create index exactly as described in doc

PUT http://localhost:9200/test_idx
{
    "index.mapper.dynamic": false
}

but got the same behavior.

Maybe I've missed something?

Thanks a lot in advance!

like image 854
Volodymyr Avatar asked Oct 21 '15 15:10

Volodymyr


People also ask

How do I turn off dynamic mapping in Elasticsearch?

You can disable dynamic mapping, both at the document and at the object level. Setting the dynamic parameter to false ignores new fields, and strict rejects the document if Elasticsearch encounters an unknown field. Use the update mapping API to update the dynamic setting on existing fields.

How do I remove a mapping field in Elasticsearch?

There is no way to delete a field from mapping. Even if you delete all documents that contain this field, this field would still be present in mapping and this is where Kibana gets its field suggestions from.

Can I change mapping for existing index Elasticsearch?

It is not possible to update the mapping of an existing field. If the mapping is set to the wrong type, re-creating the index with updated mapping and re-indexing is the only option available. In version 7.0, Elasticsearch has deprecated the document type and the default document type is set to _doc.

Can Elasticsearch index have multiple mappings?

From Elasticsearch version 6.0 by default index doesn't allow multiple types per index. The better option is to always have one document type per index. The single responsibility of the index is to maintain the single document type/mapping type per index.


1 Answers

You're almost there: the value needs to be set to strict. And the correct usage is the following:

PUT /test_idx
{
  "mappings": {
    "test_type": {
      "dynamic":"strict",
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

And pushing this a bit further, if you want to forbid the creation even of new types, not only fields in that index, use this:

PUT /test_idx
{
  "mappings": {
    "_default_": {
      "dynamic": "strict"
    },
    "test_type": {
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}

Without _default_ template:

PUT /test_idx
{
  "settings": {
    "index.mapper.dynamic": false
  },
  "mappings": {
    "test_type": {
      "dynamic": "strict",
      "properties": {
        "field1": {
          "type": "string"
        }
      }
    }
  }
}
like image 89
Andrei Stefan Avatar answered Oct 20 '22 07:10

Andrei Stefan