can I prioritize more exact matches when using ngram filter in search results?

Question

When using the ngram filter with elasticsearch so that when I search for something like "test" I return a document "latest", "tests" and "test". Is there a way to make it so that the "document exactly matching the query "test" is always returned higher up in the search results?

Answer 1

That is a bit of an issue with ngrams: you get a lot of false positives in your ranking. A solution is to combine ngrams with shingles. Basically in addition to the ngrams, you also index the full word as a separate term or even combinations of words. Shingles are basically like ngrams but with words rather than characters.

That way, an exact match against the shingle terms scores higher than something that only matches the ngrams.

Update. Here's an example of a custom analyzer. After you define it, you can use it in your mappings. In this case I use the icu_normalizer and folding and my suggestions_shingle. All this is set as the default analyzer so all my strings are handled this way.

{
    "analyzer":{
        "default":{
            "tokenizer":"icu_tokenizer",
            "filter":"icu_normalizer,icu_folding,suggestions_shingle"
        }
    },
    "filter": {
        "suggestions_shingle": {
            "type": "shingle",
            "min_shingle_size": 2,
            "max_shingle_size": 5
        }
    }
}

Answer 2

You can copy the field content to fields via the mapping. Example:

  "fullName": {
    "type": "string",
    "search_analyzer": "str_search_analyzer",
    "index_analyzer": "str_index_analyzer",
    "fields": {
        "fullWord": { "type": "string" },
        "raw": { 
            "type":  "string",
            "index": "not_analyzed"
        }
    }
  }

Note that str_index_analyzer uses nGram here. Then you can build your search to also search against these fields. Example:

{
    "query": {
      "bool": {
        "should": [{
          "multi_match": {
            "fields": [
              "firstName.fullWord",
              ...
            "query": query,
            "fuzziness": "0"
          }
        }],
        "must": [{
          "multi_match": {
            "fields": [
              "firstName",...],
            "query": query,
            "fuzziness": "AUTO"
          }
        }]
      }
    }
  };
}