Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Elasticsearch how to exclude some fields in indexing

I have a JSON data that maps automatically by the elastic search when I'm indexing the data. How can I exclude some fields in the mapping. I already tried to define the map manually but when I'm doing bulk index, It automatically maps the other fields.

ex. my JSON data looks like this

[
 {
    "id": "232",
    "name": "Lorem",
    "description": "Ipsum Dolor",
    "image": [
             {"key": "asadasd.jpg"},
             {"key": "asasd2d.jpg"}
    ],
    "is_active": true
 },
 ...  

My map when I'm defining it manually

PUT myindex
{
    "mappings": {
        "product": {
            "properties": {
                "id": { "type": "text },
                "name": { "type": "text"},
                "description": { "type": "text" },
                "is_active": { "type": "boolean" }
            }
        }
    }
}

What I want to achieve is the data still remain I just want to exclude the image property to be not included in the indexing.

So that When I query in the elastic search is still I get the data with image

Is that possible?

Thank you guys. I'm new in elasticsearch

{
    "id": "232",
    "name": "Lorem",
    "description": "Ipsum Dolor",
    "image": [
             {"key": "asadasd.jpg"},
             {"key": "asasd2d.jpg"}
    ],
    "is_active": true
 }
like image 457
Emergard Avatar asked Jul 05 '18 02:07

Emergard


People also ask

How do I select a specific field in Elasticsearch?

There are two recommended methods to retrieve selected fields from a search query: Use the fields option to extract the values of fields present in the index mapping. Use the _source option if you need to access the original data that was passed at index time.

How do I remove a mapping field in Elasticsearch?

You cannot delete the status field from this mapping. If you really need to get rid of this field, you'll have to create another mapping without status field and reindex your data.

Are all fields indexed in Elasticsearch?

By default, Elasticsearch indexes all data in every field and each indexed field has a dedicated, optimized data structure. For example, text fields are stored in inverted indices, and numeric and geo fields are stored in BKD trees.

What is reverse indexing in Elasticsearch?

An inverted index consists of all of the unique terms that appear in any document covered by the index. For each term, the list of documents in which the term appears, is stored. So essentially an inverted index is a mapping between terms and which documents contain those terms.


2 Answers

Yes, that's possible simply by adding dynamic: false to your mapping, like this:

PUT myindex
{
  "mappings": {
    "product": {
      "dynamic": false,            <-- add this line
      "properties": {
        "id": {
          "type": "text"
        },
        "name": {
          "type": "text"
        },
        "description": {
          "type": "text"
        },
        "is_active": {
          "type": "boolean"
        }
      }
    }
  }
}

The image array will still be in the source, but the mapping won't be modified.

like image 84
Val Avatar answered Sep 24 '22 17:09

Val


The problem with the accepted answer is, that you need to explicitly add mappings for all fields, which is not always wanted (e.g. for array types). You could disable the field like this:

PUT myindex
{
    "mappings": {
        "product": {
            "properties": {
                "id": { "type": "text },
                "name": { "type": "text"},
                "description": { "type": "text" },
                "is_active": { "type": "boolean" },
                "image": { "type": "object, "enabled": false }
            }
        }
    }
}

The image array is still going to be in the _source.

Reference: https://www.elastic.co/guide/en/elasticsearch/reference/current/enabled.html

like image 44
D. Phi Avatar answered Sep 24 '22 17:09

D. Phi