How can a Elasticsearch client be notified of a new indexed document?

Question

I am using Elasticsearch, and I am building a client (using the Java Client API) to export logs indexed via Logstash.

I would like to be able to be notified (by adding a listener somewhere) when a new document is index (= a new log line have been added) instead of querying the last X documents.

Is it possible ?

Answer 1

This is what you're looking for: https://github.com/ForgeRock/es-change-feed-plugin

Using this plugin, you can register to a websocket channel to receive indexation/deletion events as they happen. It has some limitations, though.

Back in the days, it was possible to install river plugins to stream documents to ES. The river feature has been removed, but this plugin above is like a "reverse river", where outside clients are notified by ES as documents get indexed.

Very useful and seemingly up-to-date with ES 6.x

UPDATE (April 14th, 2019):

According to what was said at Elastic{ON} Zurich 2019, at some point in the 7.x series, there will be a Changes API that will provide index changes notifications (document creation, update, deletion and more).