Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

spring data elasticsearch: settings and mapping config with annotations not working

I'm using embedded elasticsearch with spring boot and I was trying to use annotations for configuring settings and mappings. I followed this Tutorial which explains how to implement searching over multiple fields. Anyway, I have defined the settings.json and the mappings.json in my document entity as described here but it seems as if it is not reading the files because curling the mappings does not return the corresponding configuration as defined in the files.

Indexing is been executed by a spring batch process. It reads data from the database and writes it to elasticsearch. This works perfectly.

I get no results (should return 7 items) when I make a POST request to http://localhost:9200/profile/_search:

{
   "size": 10,
   "query": {
       "match": {
       "_all": {
           "query": "user male",
           "operator": "and"
       }
    }
  }
}

If I change the query to "user5" it returns the user with this nickname.

I will appreciate if somebody can give me a hint. What is wrong with the configuration?

The Configuration

@Configuration
@EnableElasticsearchRepositories(basePackages ="com.company.searchengine.repository")
public class ElasticSearchConfiguration {
    @Value("${elasticsearch.clustername}")
    private String esClusterName;

    @Bean
    public ElasticsearchOperations elasticsearchTemplate() throws IOException {
        return new ElasticsearchTemplate(nodeBuilder().local(true).clusterName
            (esClusterName).node()
            .client());
    }
}

The document

@EqualsAndHashCode(of = "uuid")
@ToString(exclude = "uuid")
@NoArgsConstructor(onConstructor = @__({@JsonCreator}))
@Getter
@Setter
@Document(indexName = "profiles", type = "profile", createIndex = false)
@Setting(settingPath = "/settings/settings.json")
@Mapping(mappingPath = "/mappings/mappings.json")
public class IndexedProfile {
    @Id
    @NonNull
    private String uuid;
    @NonNull
    //@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
    private String nickname;
    @NonNull
    //@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
    private String gender;
    //@Field(index = FieldIndex.not_analyzed, type = FieldType.String)
    private String about;
    private GeoPoint location;
    @Field(type = FieldType.Date, format = DateFormat.year_month_day)
    private Date birthdate;
}

The batch job

@Override
public void write(List<? extends IndexedProfile> items) throws Exception {
    List<IndexQuery> indexQueries = items.stream()
            .map(item -> new IndexQueryBuilder().withObject(item).withId(String.valueOf(item.getUuid())))
            .map(builder -> builder.withType(DOCUMENT_TYPE))
            .map(builder -> builder.withIndexName(INDEX_NAME + runId))
            .map(IndexQueryBuilder::build)
            .collect(Collectors.toList());

    this.elasticsearchTemplate.bulkIndex(indexQueries);
}

The settings

{
  "settings": {
       "analysis": {
           "filter": {
               "nGram_filter": {
                  "type": "nGram",
                   "min_gram": 2,
                   "max_gram": 20,
                   "token_chars": [
                        "letter",
                        "digit",
                        "punctuation",
                        "symbol"
                    ]
            }
        },
        "analyzer": {
             "nGram_analyzer": {
                 "type": "custom",
                 "tokenizer": "whitespace",
                 "filter": [
                      "lowercase",
                      "asciifolding",
                      "nGram_filter"
                 ]
        },
        "whitespace_analyzer": {
            "type": "custom",
            "tokenizer": "whitespace",
            "filter": [
                  "lowercase",
                  "asciifolding"
             ]
         }
      }
    }
  }
}

Mappings

{
  "mappings": {
      "profile": {
          "_all": {
               "index_analyzer": "nGram_analyzer",
               "search_analyzer": "whitespace_analyzer"
           },
           "nickname": {
                "type": "string",
                "index": "not_analyzed"
           },
           ....
       }
  }
like image 213
Rubén Avatar asked May 28 '17 17:05

Rubén


Video Answer


2 Answers

According to org.springframework.data.elasticsearch.core.mapping.SimpleElasticsearchPersistentEntity, with setting createIndex=false will cause the framework NOT to create index and mappings! So you should remove that setting or set it to true.

like image 80
evanwoods Avatar answered Oct 09 '22 18:10

evanwoods


The settings and the mappings JSON structures are not correct, try to adapt them using these reference examples:

Mappings: https://github.com/spring-projects/spring-data-elasticsearch/tree/master/src/test/resources/mappings

Settings: https://github.com/spring-projects/spring-data-elasticsearch/tree/master/src/test/resources/settings

like image 33
matteo.cajani Avatar answered Oct 09 '22 19:10

matteo.cajani