ElasticSearch date mapping

Question

I want to map the following date String as it comes out of new Date().toString()

Thu Jul 03 19:17:19 CEST 2014 but ElasticSearch gives me always the Errormessage Invalid format: "Thu Jul 03 19:17:19 CEST 2014" is malformed at "CEST 2014" and I have no idea what goes wrong. This is the mapping for the test type

{
    "properties" : {
      "postDate" : {
        "type" : "date",
        "format" : "EEE MMM dd HH:mm:ss z yyyy"
      }
    }
}

I think it has the right pattern. I even tried to increase the number of 'z', but same error. So is there a solution or do I have to find a diffrent approch.

Answer 1

You will need a different approach as the underlying date parsing software that Elasticsearch uses cannot parse "CEST" or other short format time zones, as the strings are not unique.

From the Elasticsearch documentation you can see that it uses Joda for date parsing:

The parsing of dates uses Joda. The default date parsing used if no format is specified is ISODateTimeFormat.dateOptionalTimeParser.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-date-format.html

However Joda has a problem with "CEST" and other short format time zones:

http://comments.gmane.org/gmane.comp.java.joda-time.user/1385

There is some discussion related to this but it's more focused on adding support for "CEST" for logstash as it parses and then loads data into Elasticsearch:

https://github.com/elasticsearch/logstash/pull/1378