Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to returned the analyzed fields in an ElasticSearch >2.0 search?

This question feels very similar to an old question posted here: Retrieve analyzed tokens from ElasticSearch documents, but to see if there are any changes I thought it would make sense to post it again for the latest version of ElasticSearch.

We are trying to search bodies of text in ElasticSearch with the search-query and field-mapping using the snowball stemmer built into ElasticSearch. The performance and results are great, but because we need to have the stemmed text-body for post-analysis we would like to have the search result return the actual stemmed tokens for the text-field per document in the search results.

The mapping for the field currently looks like:

      "TitleEnglish": {
        "type": "string",
        "analyzer": "standard",
        "fields": {
          "english": {
            "type": "string",
            "analyzer": "english"
          },
          "stemming": {
            "type": "string",
            "analyzer": "snowball"
          }
        }
      }

and the search query is performed specifically on TitleEnglish.stemming. Ideally I would like it to return that field, but returning that does not return the analyzed field but the original field.

Does anybody know of any way to do this? We have looked at Term Vectors, but they only seem to be returnable for individual documents or a body of documents, not for a search result?

Or perhaps other solutions like Solr or Sphinx do offer this option?


To add some extra information. If we run the following query:

GET /_analyze?analyzer=snowball&text=Eight issue of Industrial Lorestan eliminate barriers to facilitate the Committees review of

It returns the stemmed words: eight, issu, industri, etc. This is exactly the result we would like back for each matching document for all of the words in the text (so not just the matches).

like image 874
luckylwk Avatar asked Mar 16 '16 11:03

luckylwk


People also ask

How do I search all fields in Elasticsearch?

Either the query_string query or the match query would be what you're looking for. query_string will use the special _all field if none is specified in default_field , so that would work out well. And with match you can just specify the _all as well. Save this answer.

What are stored fields in Elasticsearch?

Stored fields returned as arrays For consistency, stored fields are always returned as an array because there is no way of knowing if the original field value was a single value, multiple values, or an empty array. If you need the original value, you should retrieve it from the _source field instead.


1 Answers

Unless I'm missing something evident, why not simply returning a terms aggregation on the TitleEnglish.stemming field?

{
    "query": {...},
    "aggs" : {
        "stems" : {
            "terms" : { 
                "field" : "TitleEnglish.stemming",
                "size": 50
            }
        }
    }
}

Adding that aggregation to your query, you'd get a breakdown of all the stemmed terms in the TitleEnglish.stemming sub-field from the documents that matched your query.

like image 65
Val Avatar answered Sep 19 '22 14:09

Val