Return unique results in elasticsearch

Question

I have a use case in which I have data like

{
    name: "John",
    parentid": "1234",
    filter: {a: '1', b: '3', c: '4'}
},
{
    name: "Tim",
    parentid": "2222",
    filter: {a: '2', b: '1', c: '4'}
},
{
    name: "Mary",
    parentid": "1234",
    filter: {a: '1', b: '3', c: '5'}
},
{
    name: "Tom",
    parentid": "2222",
    filter: {a: '1', b: '3', c: '1'}
}

expected results:

bucket:[{
    key: "2222",
    hits: [{
        name: "Tom" ...
    }, 
    {
        name: "Tim" ...
    }]
},
{
    key: "1234",
    hits: [{
        name: "John" ...
    },
    {
        name: "Mary" ...
    }]
}]

I want to return unique document by parentid. Although I can use top aggregation but I don't how can I paginate the bucket. As there is more chance of parentid being different than same. So mine bucket array would be large and I want to show all of them but by paginating them.

Answer 1

There is no direct way of doing this. But you can follow these steps to get desired result.

Step 1. You should know all parentid. This data can be obtained by doing a simple terms aggregation (Read more here) on field parentid and you will get only the list of parentid, not the documents matching to that. In the end you will have a smaller array on than you are currently expectig.

{
  "aggs": {
    "parentids": {
      "terms": {
        "field": "parentid",
        "size": 0 
      }
    }
  }
}

size: 0 is required to return all results. Read more here.

OR

If you already know list of all parentid then you can directly move to step 2.

Step 2. Fetch related documents by filtering documents by parentid and here you can apply pagination.

{
  "from": 0,
  "size": 20, 
  "query": {
    "filtered": {
      "query": {
        "match_all": {}
      },
      "filter": {
        "term": {
          "parentid": "2222"
        }
      }
    }
  }

}

from and size are used for pagination, so you can loop through each of parentid in the list and fetch all related documents.