ElasticSearch date conversion to Java LocalDateTime failing

Question

I have a micro service that reads data from ElasticSearch index and process it. This index has a date field and I am not able to load date field data in to document entity. Here is snapshot of what I am trying achieve.

Index

"date_created": {
  "type": "date"
}

It is storing date as

"date_created": "2015-07-02T14:56:51.000Z"

My entity class

@Data
@NoArgsConstructor
@AllArgsConstructor
@Document(indexName = "account", type = "doc")
public class Account implements Serializable {

...

    @JsonProperty("date_created")
    @JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ")
    @JsonSerialize(using = LocalDateTimeSerializer.class)
    @JsonDeserialize(using = LocalDateTimeDeserializer.class)
    private LocalDateTime dateCreated;

...

}

Error I get when I fetch records from ElasticSearch repository at run time is as below

nested exception is org.springframework.data.elasticsearch.ElasticsearchException: failed to map source ...
java.time.format.DateTimeParseException: Text '2002-08-05T04:00:00.000Z' could not be parsed, unparsed text found at index 23

Is there a way I can get around this issue? I will be trying to change index definition & JsonFormat as below, but I am not sure if that will work. I am asking in advance because I will take time to implement change in index.

Index change I will try later.

"date_created": {
  "type": "date",
  "format": "yyyy-MM-dd'T'HH:mm:ss.SSS"
}

JsonFormat change

@JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSS")

Your help is much appreciated.

Answer 1

I recreated index as mentioned above. However, after index creation, logstash started failing to pump data in index. Fixing data at instream could have been costly. Hence I restored to original index back.

What really solved my problems? I did a minor change in pattern of @JsonFormat and it did the trick.

Before

@JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ")

After

@JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")

So, basically the 'Z' in the date that was being returned was a character and not a date formatter. With single quotes, I told formatter to treat it as a character and NOT a format string.

Hope this solves confusion for others.

Answer 2

I was experiencing the same issue: 'Z' in the date value is treated as a character, so the date parse failed. My solution is a little bit different though:

@Field(type = FieldType.Date, format = DateFormat.custom, pattern = "yyyy-MM-dd'T'HH:mm:ss'Z'")
private LocalDateTime dateField;

And if we don't want to repeat the pattern on different fields again and again, we can try to centralize to the conversion logic. Spring Data Elastic Search provide the custom conversion feature, check here for the example. We can write a converter from String to Date, and put the date pattern over there.