Elasticsearch _timestamp

Question

I tried to define the _timestamp property on an index. So first, I create the index

curl -XPUT 'http://elasticsearch:9200/ppe/'

response from the server : {"ok":true,"acknowledged":true}

then I tried to define the mapping with a _timestamp

curl -Xput 'http://elasticsearch:9200/ppe/log/_mapping' -d '{
  "log": {
    "properties": {
      "_ttl": {
        "enabled": true
      },
      "_timestamp": {
        "enabled": true,
        "store": "yes"
      },
      "message": {
        "type": "string",
        "store": "yes"
      },
      "appid": {
        "type": "string",
        "store": "yes"
      },
      "level": {
        "type": "integer",
        "store": "yes"
      },
      "logdate": {
        "type": "date",
        "format": "date_time_no_millis",
        "store": "yes"
      }
    }
  }
}'

and I receive as answer from the server

{
  "error": "MapperParsingException[No type specified for property [_timestamp]]",
  "status": 400
}

What's wrong with my mapping?

Answer 1

Special fields such as _ttl and _timestamp have to be defined on the same level as the properties object:

curl -Xput 'http://elasticsearch:9200/ppe/log/_mapping' -d '{
    "log": {
        "_ttl": {
            "enabled": true
        },
        "_timestamp": {
            "enabled": true,
            "store": "yes"
        },
        "properties": {
            "message": {
                "type": "string",
                "store": "yes"
            },
            "appid": {
                "type": "string",
                "store": "yes"
            },
            "level": {
                "type": "integer",
                "store": "yes"
            },
            "logdate": {
                "type": "date",
                "format": "date_time_no_millis",
                "store": "yes"
            }
        }
    }
}
'

Answer 2

Note though that although _timestamp is defined on top level it will be returned inside fields:

curl 'http://localhost:9200/myindex/mytype/AUqL0PW7YDMmKSIKO1bk?pretty=true&fields=_timestamp'
{
  "_index" : "myindex",
  "_type" : "mytype",
  "_id" : "AUqL0PW7YDMmKSIKO1bk",
  "_version" : 1,
  "found" : true,
  "fields" : {
    "_timestamp" : 1419684935099
  }
}

Note that _timestamp must be explicitly requested by fields=_timestamp or fields=_timestamp,_source.

Note that _timestamp can be returned only when this field is marked as 'store': true. But there is a way to access this value when sorting by _timestamp, like this:

curl 'http://localhost:9200/myindex/mytype/_search?pretty=true' -d ' 
   { "sort": [ "_timestamp" ], "size": 1}
 '

Gives result:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : null,
    "hits" : [ {
       "_index" : "myindex",
       "_type" : "mytype",
       "_id" : "AUqL0PDXYDMmKSIKO1bj",
       "_score" : null,
       "sort" : [ 1419684933847 ]
     } ]
  }
}

And now sort[0] is the value for the first (and the only in this case) sort value: _timestamp. _timestamp does not have to be marked as "store": true when used in this manner.