Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ElasticSearch term aggregation

I'm trying to perform a term aggregation using elastic search for the data below with following query, the output breaks the names into tokens (see output below). So I tried mapping the os_name as multi_field and now I am not able to query by it. Is it possible to have index without tokens? such as "Fedora Core"?

Query:

GET /temp/example/_search
{
  "size": 0,
  "aggs": {
     "OS": {
       "terms": {
           "field": "os_name"
       }
     }
  }
}

Data:

...
    {
        "_index": "temp",
        "_type": "example",
        "_id": "3",
        "_score": 1,
        "_source": {
           "title": "system3",
           "os_name": "Fedora Core",
           "os_version": 18
        }
     },
     {
        "_index": "temp",
        "_type": "example",
        "_id": "1",
        "_score": 1,
        "_source": {
           "title": "system1",
           "os_name": "Fedora Core",
           "os_version": 20
        }
     },
     {
        "_index": "temp",
        "_type": "example",
        "_id": "2",
        "_score": 1,
        "_source": {
           "title": "backup",
           "os_name": "Yellow Dog",
           "os_version": 6
        }
     }
...

Output:

       ...
        {
           "key": "core",
           "doc_count": 2
        },
        {
           "key": "fedora",
           "doc_count": 2
        },
        {
           "key": "dog",
           "doc_count": 1
        },
        {
           "key": "yellow",
           "doc_count": 1
        }
       ...

mapping:

PUT /temp
{
  "mappings": {
    "example": {
      "properties": {
        "os_name": {
          "type": "string"
        },
        "os_version": {
          "type": "long"
        },
        "title": {
          "type": "string"
        }
      }
    }
  }
}
like image 792
codeBarer Avatar asked May 20 '14 20:05

codeBarer


2 Answers

Actually you should change your mapping like this

"os_name": {
  "type": "string",
  "fields": {
     "raw": {
        "type": "string",
        "index": "not_analyzed"
     }
  }
},

and your aggs should be changed to:

GET /temp/example/_search
{
  "size": 0,
  "aggs": {
     "OS": {
       "terms": {
           "field": "os_name.raw"
       }
     }
  }
}
like image 90
mst Avatar answered Oct 23 '22 04:10

mst


One solution that would work is to set the field to not_analyzed (Read more about it in the docs for attribute "index").

This solution will not analyze the input at all, depending on your requirements you might wish to set a custom analyzer, e.g. to not split the words, but lowercase them, to get case insensitive results.

curl -XDELETE localhost:9200/temp
curl -XPUT localhost:9200/temp -d '
{
  "mappings": {
    "example": {
      "properties": {
        "os_name": {
          "type": "string",
          "index" : "not_analyzed"
        },
        "os_version": {
          "type": "long"
        },
        "title": {
          "type": "string"
        }
      }
    }
  }
}'

curl -XPUT localhost:9200/temp/example/1 -d '
{
    "title": "system3",
    "os_name": "Fedora Core",
    "os_version": 18
}'

curl -XPUT localhost:9200/temp/example/2 -d '
{
    "title": "system1",
    "os_name": "Fedora Core",
    "os_version": 20
}'

curl -XPUT localhost:9200/temp/example/3 -d '
{
    "title": "backup",
    "os_name": "Yellow Dog",
    "os_version": 6
}'

curl -XGET localhost:9200/temp/example/_search?pretty=true -d '
{
  "size": 0,
  "aggs": {
     "OS": {
       "terms": {
           "field": "os_name"
       }
     }
  }
}'

Output:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "OS" : {
      "buckets" : [ {
        "key" : "Fedora Core",
        "doc_count" : 2
      }, {
        "key" : "Yellow Dog",
        "doc_count" : 1
      } ]
    }
  }
}
like image 22
Thorsten Avatar answered Oct 23 '22 02:10

Thorsten