Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Elasticsearch Java API addMapping() and setSettings() usage

Problem: How to create an index from a json file using

The json file contains a definition for the index de_brochures. It also defines an analyzer de_analyzerwith custom filters that are used by the respective index. As the json works with curl and Sense I assume I have to adapt the syntax of it to work with the java API.

I don't want to use XContentFactory.jsonBuilder() as the json comes from a file!

I have the following json file to create my mapping from and to set settings:

Using Sense with PUT /indexname it does create an index from this.

{
  "mappings": {
    "de_brochures": {
      "properties": {
        "text": {
          "type": "string",
          "store": true,
          "index_analyzer": "de_analyzer"
        },
        "classification": {
          "type": "string",
          "index": "not_analyzed"
        },
        "language": {
          "type": "string",
          "index": "not_analyzed"
        }
      }
    }
  "settings": {
    "analysis": {
      "filter": {
        "de_stopwords": {
          "type": "stop",
          "stopwords": "_german_"
        },
        "de_stemmer": {
          "type": "stemmer",
          "name": "light_german"
        }
      },
      "analyzer": {
        "de_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "de_stopwords",
            "de_stemmer"
          ]
        }
      }
    }
  }
}

As the above did not work with addMapping() alone I tried to split it into two seperate files (I realized that I had to remove the "mappings": and "settings": part):

------ Mapping json ------
{
  "de_brochures": {
    "properties": {
      "text": {
        "type": "string",
        "store": true,
        "index_analyzer": "de_analyzer"
      },
      "classification": {
        "type": "string",
        "index": "not_analyzed"
      },
      "language": {
        "type": "string",
        "index": "not_analyzed"
      }
    }
  }
}
------- Settings json --------
{
  "analysis": {
    "filter": {
      "de_stopwords": {
        "type": "stop",
        "stopwords": "_german_"
      },
      "de_stemmer": {
        "type": "stemmer",
        "name": "light_german"
      }
    },
    "analyzer": {
      "de_analyzer": {
        "type": "custom",
        "tokenizer": "standard",
        "filter": [
          "lowercase",
          "de_stopwords",
          "de_stemmer"
        ]
      }
    }
  }
}

This is my java code to load and add/set the json.

CreateIndexRequestBuilder createIndexRequestBuilder = client.admin().indices().prepareCreate(index);
// CREATE SETTINGS
String settings_json = new String(Files.readAllBytes(brochures_mapping_path));
createIndexRequestBuilder.setSettings(settings_json);
// CREATE MAPPING
String mapping_json = new String(Files.readAllBytes(brochures_mapping_path));
createIndexRequestBuilder.addMapping("de_brochures", mapping_json);
CreateIndexResponse indexResponse = createIndexRequestBuilder.execute().actionGet();

There is no more complaint about the mapping file's structure but it now fails with the error:

Caused by: org.elasticsearch.index.mapper.MapperParsingException: Analyzer [de_analyzer] not found for field [text]
like image 844
Peter Avatar asked Mar 29 '15 15:03

Peter


1 Answers

Solution: I managed to do it with my original json file using createIndexRequestBuilder.setSource(settings_json);

like image 189
Peter Avatar answered Oct 05 '22 05:10

Peter