Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

elasticsearch - querying multiple indexes is possible?

I have an elasticsearch cluster with 3 indexes:

/users/user /events/visit /events/register /pages/page 

So, now I need to run queries processing multiple indexes.

Eg: Get gender of users registered in page X. (To get this info, I need infos from multiple indexes.)

Is this possible? Maybe integrating hadoop?

like image 702
user3175226 Avatar asked Jun 19 '14 20:06

user3175226


People also ask

Can we query from two indexes in Elasticsearch?

To search multiple data streams and indices, add them as comma-separated values in the search API's request path. The following request searches the my-index-000001 and my-index-000002 indices. You can also search multiple data streams and indices using an index pattern.

Can a query use multiple indexes?

Yes, MySQL can use multiple index for a single query. The optimizer will determine which indexes will benefit the query. You can use EXPLAIN to obtain information about how MySQL executes a statement.

How many indexes can Elasticsearch handle?

Indexes themselves have no limit, however shards do, the recommended amount of shards per GB of heap is 20(JVM heap - you can check on kibana stack monitoring tab), this means if you have 5GB of JVM heap, the recommended amount is 100.

Can you have multiple indexes?

It is possible for an index to have two or more columns. Multi column indexes are also known as compound or concatenated indexes. Let us look at a query that could use two different indexes on the table based on the WHERE clause restrictions.


2 Answers

This is quite easy within Elasticsearch itself! Anytime you would specify an index, you can separate additional indices by comma.

curl -XGET 'http://localhost:9200/index1,index2/_search?q=yourQueryHere' 

You can also search all indices with _all.

curl -XGET 'http://localhost:9200/_all/_search?q=yourQueryHere' 

Here's some helpful documentation from elasticsearch's website. This site has TONS of info, but it is a bit difficult to find sometimes, IMO.

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-index.html

like image 119
Brian Mego Avatar answered Sep 21 '22 14:09

Brian Mego


By not limiting our search to a particular index or type, we have searched across all documents in the cluster. Elasticsearch forwarded the search request in parallel to a primary or replica of every shard in the cluster.

       1)/users,events,pages/_search : Search all types in the users,events and pages         2)/u*,e*,p*/_search : Search all types in any indices beginning with u,e or beginning with p         3)/events/visit,register/_search : Search types visit and register in the events index         4) /_all/user,visit,register,page/_search : Search types users,events and pages in specified indices 
like image 25
Pavan Kumar Varma Avatar answered Sep 21 '22 14:09

Pavan Kumar Varma