Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Elasticsearch - how to return only data, not meta information?

Tags:

When doing a search, Elasticsearch returns a data structure that contains various meta information.

The actual result set is contained within a "hits" field within the JSON result returned from the database.

Is it possible for Elasticsearch to return only the needed data (the contents of then "hits" field) without being embedded within all the other meta data?

I know I could parse the result into JSON and extract it, but I don't want the complexity, hassle, performance hit.

thanks!

Here is an example of the data structure that Elasticsearch returns.

{     "_shards":{         "total" : 5,         "successful" : 5,         "failed" : 0     },     "hits":{         "total" : 1,         "hits" : [             {                 "_index" : "twitter",                 "_type" : "tweet",                 "_id" : "1",                  "_source" : {                     "user" : "kimchy",                     "postDate" : "2009-11-15T14:12:12",                     "message" : "trying out Elastic Search"                 }             }         ]     } } 
like image 970
Duke Dougal Avatar asked Jun 02 '12 07:06

Duke Dougal


People also ask

How do I retrieve data from Elasticsearch?

You can use the search API to search and aggregate data stored in Elasticsearch data streams or indices. The API's query request body parameter accepts queries written in Query DSL. The following request searches my-index-000001 using a match query. This query matches documents with a user.id value of kimchy .

How do I get Elasticsearch to index all data?

elasticsearch(ES) supports both a GET or a POST request for getting the data from the ES cluster index. I would suggest to use a UI plugin with elasticsearch http://mobz.github.io/elasticsearch-head/ This will help you get a better feeling of the indices you create and also test your indices.

What is _meta in Elasticsearch?

These are not used at all by Elasticsearch, but can be used to store application-specific metadata, such as the class that a document belongs to: PUT my-index-000001 { "mappings": { "_meta": { "class": "MyApp::User", "version": { "min": "1.0", "max": "1.3" } } } } Console.


1 Answers

You can at least filter the results, even if you cannot extract them. The "common options" page of the REST API explains the "filter_path" option. This lets you filter only the portions of the tree you are interested in. The tree structure is still the same, but without the extra metadata.

I generally add the query option:

&filter_path=hits.hits.*,aggregations.* 

The documentation doesn't say anything about this making your query any faster (I doubt that it does), but at least you could return only the interesting parts.

  • Corrected to show only hits.hits.*, since the top level "hits" has metadata as well.
like image 136
Michael Erickson Avatar answered Nov 16 '22 17:11

Michael Erickson