Spring Data Elasticsearch's @Field annotation not working

Question

I have a Spring Boot application with Spring Data Elasticsearch plugin in the pom.xml. I created a document class which i'd like to index:

@Document(indexName = "operations", type = "operation")
public class OperationDocument {

@Id
private Long id;

@Field(
    type = FieldType.String, 
    index = FieldIndex.analyzed, 
    searchAnalyzer = "standard", 
    indexAnalyzer = "standard",
    store = true
)
private String operationName;

@Field(
    type = FieldType.Date, 
    index = FieldIndex.not_analyzed, 
    store = true, 
    format = DateFormat.custom, pattern = "dd.MM.yyyy hh:mm"
)
private Date dateUp;

@Field(
    type = FieldType.String, 
    index = FieldIndex.not_analyzed, 
    store = false
) 
private String someTransientData;

@Field(type = FieldType.Nested)
private List<Sector> sectors;

//Getter and setters

I also created a repository for this class:

 public interface OperationDocumentRepository 
      extends ElasticsearchRepository<OperationDocument, Long> {
 }

I made a test that indexes three sample objects using the repository. It's pretty long so I'll post it only is needed. The fact is that the mapping created in the ES server ignores configuration set by @Field annotations:

"mappings": {
  "operation": {
    "properties": {
      "operationName": {
        "type": "string"
      },
      "dateUp": {
        "type": "long"
      },
      "someTransientData": {
        "type": "string"
      },
      "sectors": {
        "properties": {
          "id": {
            "type": "long"
          },
          "sectorName": {
            "type": "string"
          }
        }
      }
    }
  }
}

There is no info about analyzers, "someTransientData" is stored and indexed, and dateUp is typed as Long instead of Date.

A sample document requested directly from the server:

 {
   "_index": "operations",
   "_type": "operation",
   "_id": "AUyUk2cY3nXeOFxdOlQW",
   "_version": 1,
   "_score": 1,
   "_source": {
     "id": null,
     "operationName": "Second Operation Name",
     "dateUp": 1428421827091,
     "someTransientData": "Do not index or store",
     "sectors": [
       {
         "id": 2,
         "sectorName": "Health Care"
       },
       {
         "id": 3,
         "sectorName": "Construction"
       }
     ]
   }
 }

I also noted that when I run the application for the second time, at startup time I get this error, only printed when the index already exists:

ERROR 19452 --- [main] .d.e.r.s.AbstractElasticsearchRepository : failed to load elasticsearch nodes : org.elasticsearch.index.mapper.MergeMappingException: Merge failed with failures {[mapper [someTransientData] has different index values, mapper [someTransientData] has different tokenize values, mapper [someTransientData] has different index_analyzer, object mapping [sectors] can't be changed from non-nested to nested, mapper [operationName] has different store values, mapper [operationName] has different index_analyzer, mapper [dateUp] of different type, current_type [long], merged_type [date]]}

It's this a bug of Spring Data Elastic Search or I'm doing something wrong?

I tried the stable version provided by spring boot and last snapshot of spring-data-elasticsearch. I also tried the embedded Elasticsearch server provided by the plugin and an external one of the current version. I got always the same results.

Answer 1

I could finally replicate and solve the problem. The fact is that I was using ElasticTemplate for indexing and searching docs instead of repositories, because my business logic got a more complicated (use of aggregations, etc.).

After that, I removed the unused OperationDocumentRespository. It seems that the repository is needed for the type mapping being posted to ES server on startup. I thought having the @Document class should be enough, but it isn't.

So we have two options here:

• Keep the OperationDocumentRepository

• Add this line to the app startup:

  elasticsearchTemplate.putMapping(OperationDocument.class);
  

Answer 2

I tried to replicate issue using spring data elasticsearch sample application but as per your configuration i am getting the desired results as mentioned above.

Entire code is committed to project here --> link

Look at the TestCase which generate index and apply mapping when spring context loaded--> link

Here is mapping generated by TestCase :

  {
  "operations" : {
    "aliases" : { },
    "mappings" : {
      "operation" : {
        "properties" : {
          "dateUp" : {
            "type" : "date",
            "store" : true,
            "format" : "dd.MM.yyyy hh:mm"
          },
          "operationName" : {
            "type" : "string",
            "store" : true,
            "analyzer" : "standard"
          },
          "sectors" : {
            "type" : "nested"
          },
          "someTransientData" : {
            "type" : "string",
            "index" : "not_analyzed"
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "refresh_interval" : "1s",
        "number_of_shards" : "5",
        "store" : {
          "type" : "fs"
        },
        "creation_date" : "1428677234773",
        "number_of_replicas" : "1",
        "version" : {
          "created" : "1040499"
        },
        "uuid" : "-djzLu-IQ0CBs-M6R0-R6Q"
      }
    },
    "warmers" : { }
  }
}

Can you create similar example with spring boot using https://github.com/spring-projects/spring-boot/tree/master/spring-boot-samples/spring-boot-sample-data-elasticsearch

and commit on public share?