Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to list unique values of a particular field in Kibana

I am having a field named rpc in my elasticsearch database and I am displaying it using Kibana. When I search in search bar of kibana like:

rpc:*

It display all the values of rpc field but I want to have only those value to be displayed which are unique.

like image 912
tushan Avatar asked Jul 13 '15 14:07

tushan


People also ask

How do I get distinct values of a field in Elasticsearch?

You can user terms aggregation to get distinct values from your _source. As you have mentioned you don't want any other data from _source hence you can give size=0. This will give you all unique Gender values with their count in the response.


3 Answers

I have been playing around with Kibana4 since a couple of weeks now. I find it intuitive and simple and the experience has been great till now. Following your question, I tried getting unique results via a Data Table visualization. Why? Because I personally find it easier to understand. Following are the steps:

1. Get unique count

Create the visualization (Visualize -> Data Table). First lets get the count of how many unique entries we have for a particular field (We will use this in the later part for verification). I'm using clientip.raw but as I see, it will work just fine with any friendly field name too.

2. Set the aggregation right

Set you aggregation back to count and have a Split Rows as follows. Not doing this will give you count 1 for each field value (since it is looking for unique counts) when you populate the table. Noteworthy part is setting the Top field to 0. Because Kibana won't let you enter anything else than a digit (Obviously!). This was the tricky part. Hit Apply and you'll get the results. Unique field values and the count of each of them.

3. Verification:

Going to the last page of the table, we see there are exactly 543 results. This is how I know it works.

What Next?

You save this visualization and add it to a Dashboard. There you can always check the request, query, response and other stats.

like image 135
mathakoot Avatar answered Oct 16 '22 09:10

mathakoot


Just an addition to the above mathakoot answer.

For the user of newer version (which do not allow bucket size of 0 anymore) just set a value greater than the maximum number of result

enter image description here

And report the value in the Options>Per Page field

enter image description here

like image 24
Gab Avatar answered Oct 16 '22 10:10

Gab


I am using Kibana 6 so the UI looks a bit different than the older answers here.

Here is what worked for me

  1. Create a visualization from your query, I used a line graph type (don't think it matters)
  2. Under Data, set metrics aggregation = "Unique Count" and set field to your field.
  3. Set x-axis aggregation = "Terms" and set field to your field.
  4. Set Size > your number of records
  5. Under Metrics and Axes, disable drawing of the graph, circles, and labels (this really helps the UI not lag)
  6. Run query and then click "Inspect" and download CSV
Data Metrics & Axes
enter image description here enter image description here
like image 1
Skylar Brown Avatar answered Oct 16 '22 11:10

Skylar Brown