Elasticsearch global search different filter on multiple indexes

Question

We have got multiple indices in Elastic Search and would like to search the data across all indices, but we want to apply different filters on different indices.

For example:

• few indices depends on client_id, hence a client_id filter is required
• we have is_deleted flag in few indexes, hence is_deleted filter is required

How should one approach this in Elastic Search?

Also, we are using highlight feature, which is supposed to give suggestions to the users. But we would like to ignore certain fields in the highlighted results. Is it possible to exclude certain fields at global level?

Answer 1

That's possible, using a filtered queries, nested within a boolean query.

This example illustrates the basic setup (notice how different filters are used):

 @results = elastic_client.search([:dogs, :cats], {
   :bool => {
     :should => [
       # cats
       {
         :filtered => {
           :query => {
             :multi_match => {
               :query => 'meow', # repeated, replace with a variable
               :type => 'phrase_prefix',
               :fields => ['name', 'age']
             }
           },
           :filter => {
             :and => [
               { :term => { :owner_id => '123' } },
               { :type => { :value => 'cat' } }
             ]
           }
         }
       },
       # dogs
       {
         :filtered => {
           :query => {
             :multi_match => {
               :query => 'meow', # repeated, replace with a variable
               :type => 'phrase_prefix',
               :fields => ['name', 'color']
             }
           },
           :filter => {
             :and => [
               { :term => { :kennel_id => '456' } },
               { :type => { :value => 'dog' } }
             ]
           }
         }
       }
     ]
   }
 })

This particular code may or may not work with your ES-client, but it should give a fairly good idea of the concept.

Note that the query "meow" occurs twice, and you may want to use a variable instead, to search for the same thing in the two indices. Also, multi_match could be some other type of query obviously.