Function score query with field_value_factor on not (yet) existing field

Question

I've been messing around with this problem for quite some time now and can't get round to fixing this.

Take the following case:

I have 2 employees in my company which have their own blog page:

POST blog/page/1
{
  "author": "Byron",
  "author-title": "Junior Software Developer",
  "content" : "My amazing bio"
}

and

POST blog/page/2
{
  "author": "Jason",
  "author-title": "Senior Software Developer",
  "content" : "My amazing bio is better"
}

After they created their blog posts, we would like to keep track of the 'views' of their blogs and boost search results based on their 'views'.

This can be done by using the function score query:

GET blog/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "author-title": "developer"
        }
      },
      "functions": [
        {
          "filter": {
            "range": {
              "views": {
                "from": 1
              }
            }
          }, 
          "field_value_factor": {
            "field": "views"
          }
        }
      ]
    }
  }
}

I use the range filter to make sure the field_value_factor doesn't affect the score when the amount of views is 0 (score would be also 0).

Now when I try to run this query, I will get the following exception:

nested: ElasticsearchException[Unable to find a field mapper for field [views]]; }]

Which makes sense, because the field doesn't exist anywhere in the index. If I were to add views = 0 on index-time, I wouldn't have the above issue as the field is known within the index. But in my use-case I'm unable to add this either on index-time or to a mapping.

Based on the ability to use a range filter within the function score query, I thought I would be able to use a exists filter to make sure that the field_value_factor part would only be executed when the field is actually present in the index, but no such luck:

GET blog/_search
{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "author-title": "developer"
        }
      },
      "functions": [
        {
          "filter": {
            "bool": {
              "must": [
                {
                  "exists": {
                    "field": "views"
                  }
                },
                {
                  "range": {
                    "views": {
                      "from": 1
                    }
                  }
                }
              ]
            }
          },
          "field_value_factor": {
            "field": "views"
          }
        }
      ]
    }
  }
}

Still gives:

nested: ElasticsearchException[Unable to find a field mapper for field [views]]; }]

Where I'd expect Elasticsearch to apply the filter first, before parsing the field_value_factor.

Any thoughts on how to fix this issue, without the use of mapping files or fixing during index-time or scripts??

Answer 1

The error you're seeing occurs at query parsing time, i.e. nothing has been executed yet. At that time, the FieldValueFactorFunctionParser builds the filter_value_factor function to be executed later, but it notices that the views field doesn't exist in the mapping type.

Note that the filter has not been executed yet, just like the filter_value_factor function, it has only been parsed by FunctionScoreQueryParser.

I'm wondering why you can't simply add a field in your mapping type, it's as easy as running this

curl -XPUT 'http://localhost:9200/blog/_mapping/page' -d '{
    "page" : {
        "properties" : {
            "views" : {"type" : "integer"}
        }
    }
}'

If this is REALLY not an option, another possibility would be to use script_score instead, like this:

{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "author-title": "developer"
        }
      },
      "functions": [
        {
          "filter": {
            "range": {
              "views": {
                "from": 1
              }
            }
          }, 
          "script_score": {
            "script": "_score * doc.views.value"
          }
        }
      ]
    }
  }
}