Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the most suitable datatype to use in aggregations with ElasticSearch 5: numeric or keyword?

In a Elasticsearch index, I have a few fields which are referencing main categories' ids (e.g. sector_id, country_id, etc...).

These fields are solely use for filtering (using the term/terms filters) and for creating buckets in terms aggregations (among others).

Each one of them is currently using the smallest suitable numeric datatype (e.g. byte, short, etc..)

Is this the best datatype to be used on these for heavy aggregations?

Or should these be using the keyword datatype?

Thanks in advance for any advice!

like image 895
Concordia Discors Avatar asked Nov 24 '16 09:11

Concordia Discors


People also ask

Is Elasticsearch good for aggregations?

Elasticsearch Aggregations provide you with the ability to group and perform calculations and statistics (such as sums and averages) on your data by using a simple search query. An aggregation can be viewed as a working unit that builds analytical information across a set of documents.

What is data type in Elasticsearch?

Elasticsearch supports a number of different data types for the fields in a document: Core data types: String, Date, Numeric (long, integer, short, byte, double, and float), Boolean, Binary. Complex data types: Array: Array support does not require a dedicated type. Object: Object for single JSON objects.

What is the difference between text and keyword in Elasticsearch?

The crucial difference between them is that Elasticsearch will analyze the Text before it's stored into the Inverted Index while it won't analyze Keyword type. Analyzed or not analyzed will affect how it will behave when getting queried.


1 Answers

If the values of those fields are numeric, you should go for a numeric type, if they are strings, then go for the keyword type.

One thing to bear in mind is that if you want to run range queries and/or range aggregations on those fields at some point, you should prefer using a numeric type up front so that those values can be sorted numerically and not lexically.

For instance: if you have country ids such as 1, 2, 3, ..., 10, 11, 12, ..., 20, ... and they are mapped as keyword (i.e. string) then if you run a range query (or aggregation) on them with from: 1, to: 3, you'll also get 11, 12, 13, etc since in the string world, 11 is lexically smaller than 3.

like image 184
Val Avatar answered Oct 19 '22 17:10

Val