AND query in elasticsearch with curl

Question

Let's inject some data in Elasticsearch

curl -XPUT 'localhost:9200/customer/external/1' -d '{ "author": "John", "published_from":"2016-08-03" }'
curl -XPUT 'localhost:9200/customer/external/2' -d '{ "author": "Jeanne", "published_from":"2016-08-03" }'
curl -XPUT 'localhost:9200/customer/external/3' -d '{ "author": "Jean", "published_from":"2016-08-05" }'

I am trying to query document with published_from=2016-08-03 and author=John. I try to do it with this curl command:

curl -g "localhost:9200/customer/external/_search?pretty&filter_path=hits.hits._source.author&q=+author:John+published_from:2016-08-03"

Yet, the output displays Jeanne

{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "author" : "John"
        }
      },
      {
        "_source" : {
          "author" : "Jeanne"
        }
      }
    ]
  }
}

When I try this curl command :

curl "localhost:9200/customer/external/_search?pretty&filter_path=hits.hits._source.author&q=%2Bauthor%3AJohn+%2Bpublished_from%3A2016-08-03"

The output is exactly what I want.

{
  "hits" : {
    "hits" : [
      {
        "_source" : {
          "author" : "John"
        }
      }
    ]
  }
}

Why is the first command not working as expected ?

Answer 1

The + signs in the first URL:

...&q=+author:John+published_from:2016-08-03

are interpreted (on server-side), in accordance to the percent-encoding rules as spaces. The space is usually encoded as %20, but for historical reasons, + is also a valid encoding of the space character.

That means the query string that ElasticSearch gets looks like:

author:John published_from:2016-08-03

According to query string syntax, it will find any document that contains one or more of author:John or published_from:2016-08-03.

When you properly encode the Elastic's + operator (in the second URL), the query received is:

+author:John +published_from:2016-08-03

Note the + was decoded as space , and %2B as +.

Simple query parameter encoding in curl

To avoid manual (percent-)encoding of queries, you can use the --data-urlencode option, and provide raw key=value pairs.

For example:

curl -G 'localhost:9200/customer/external/_search?pretty&filter_path=hits.hits._source.author' --data-urlencode 'q=+author:John +published_from:2016-08-03'

Here curl will combine the query parameters from the URL with those provided with the -d/--data-urlencode options. We need -G to force a GET request, since -d/--data-urlencode defaults to a POST request.