I'm using Elasticsearch's multi search to run a list of queries through the python library. It seems to be working... mostly. The problem is that if I send up 200 queries, it returns 100 results (no errors). I've tried running even 20 queries and still get 10 results. For some reason it seems to always return only half of the queries. Which also means I don't know which query the results correlate to.
The queries all work individually but would take too long. There's no mention in the docs of the possibility of the response not containing the same number of queries so I'm not sure where to go here. I've stepped into the code as much as the python library will let me to verify all of the queries are being sent up and it seems to work as expected.
Thanks for any help or a push in the right direction!
Edit: Here's an example of the query I use.
{
"filter":{
"and":[
{
"not":{
"term":{
"uuid":"60507f9e-01c1-11e5-a369-34363bd16ec4"
}
}
},
{
"term":{
"brand_id":22212
}
}
]
},
"query":{
"bool":{
"minimum_should_match":1,
"should":[
{
"match":{
"name":{
"query":"spork",
"boost":3
}
}
},
{
"match":{
"categories":{
"slop":10,
"type":"phrase",
"query":"household plates, bowls, cups & flatware"
}
}
}
],
"must_not":[
],
"must":[
{
"match":{
"name_analyzed":{
"boost":4,
"query":"spork",
"type":"phrase",
"slop":15
}
}
}
]
}
}
}
I create a list of however many queries like these I'd like to run and execute them as so, as per the python elasticsearch documentation:
elasticsearch.msearch(list_of_search_queries)
The reason might be that you haven't provided the size parameter in the query. This limits the result count to 10 by default. Out of all the results the top 10 might be from the two index even thought the match is present in third index as well.
Elasticsearch DSL is a high-level library whose aim is to help with writing and running queries against Elasticsearch. It is built on top of the official low-level client ( elasticsearch-py ). It provides a more convenient and idiomatic way to write and manipulate queries.
Whew! That was annoying.
So the caveat between search and msearch is that elasticsearch expects two lines per query. The first line indicates the index (can be left blank if you've already specified it) and the second line is your actual query.
So when creating my list of queries, I just append {} right before it as so:
list_of_queries = []
for thing in list_of_things:
list_of_queries.append({})
list_of_queries.append(thing.get_query())
Then it works!
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With