Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to make elasticsearch add the timestamp field to every document in all indices?

Elasticsearch experts,

I have been unable to find a simple way to just tell ElasticSearch to insert the _timestamp field for all the documents that are added in all the indices (and all document types).

I see an example for specific types: http://www.elasticsearch.org/guide/reference/mapping/timestamp-field/

and also see an example for all indices for a specific type (using _all): http://www.elasticsearch.org/guide/reference/api/admin-indices-put-mapping/

but I am unable to find any documentation on adding it by default for all documents that get added irrespective of the index and type.

like image 263
Gautam M Avatar asked Jun 16 '13 17:06

Gautam M


People also ask

How do you add a timestamp in Elasticsearch?

If you're running Elasticsearch version 6.5 or newer, you can use the index. default_pipeline settings to create a timestamp field for an index. This can be accomplished by using the Ingest API and creating a pipeline at the time your index is created.

What is the Elasticsearch query to get all documents from an index?

Elasticsearch will get significant slower if you just add some big number as size, one method to use to get all documents is using scan and scroll ids. The results from this would contain a _scroll_id which you have to query to get the next 100 chunk. This answer needs more updates. search_type=scan is now deprecated.

What is the data structure called in Elasticsearch that contains all of the documents that a user wishes to store and organize?

Elasticsearch uses a data structure called an inverted index that supports very fast full-text searches. An inverted index lists every unique word that appears in any document and identifies all of the documents each word occurs in.

How do you update existing field mapping in Elasticsearch?

It is not possible to update the mapping of an existing field. If the mapping is set to the wrong type, re-creating the index with updated mapping and re-indexing is the only option available. In version 7.0, Elasticsearch has deprecated the document type and the default document type is set to _doc.


1 Answers

Elasticsearch used to support automatically adding timestamps to documents being indexed, but deprecated this feature in 2.0.0

From the version 5.5 documentation:

The _timestamp and _ttl fields were deprecated and are now removed. As a replacement for _timestamp, you should populate a regular date field with the current timestamp on application side.

like image 169
FluxLemur Avatar answered Sep 23 '22 21:09

FluxLemur