Paging results of aggregation pipeline with spring data mongodb

Question

I am having a bit of trouble with paging the results of an aggregation pipeline. After looking at In spring data mongodb how to achieve pagination for aggregation I came up with what feels like a hacky solution. I first performed the match query, then grouped by the field that I searched for, and counted the results, mapping the value to a private class:

private long getCount(String propertyName, String propertyValue) {
    MatchOperation matchOperation = match(
        Criteria.where(propertyName).is(propertyValue)
    );
    GroupOperation groupOperation = group(propertyName).count().as("count");
    Aggregation aggregation = newAggregation(matchOperation, groupOperation);
    return mongoTemplate.aggregate(aggregation, Athlete.class, NumberOfResults.class)
        .getMappedResults().get(0).getCount();
}

private class NumberOfResults {
    private int count;

    public int getCount() {
        return count;
    }

    public void setCount(int count) {
        this.count = count;
    }
}

This way, I was able to provide a "total" value for the page object I was returning:

public Page<Athlete> findAllByName(String name, Pageable pageable) {
    long total = getCount("team.name", name);
    Aggregation aggregation = getAggregation("team.name", name, pageable);
    List<Athlete> aggregationResults = mongoTemplate.aggregate(
        aggregation, Athlete.class, Athlete.class
    ).getMappedResults();
    return new PageImpl<>(aggregationResults, pageable, total);
}

You can see that the aggregation to get the total count of results is not too different from the actual aggregation that I want to perform:

MatchOperation matchOperation = match(Criteria.where(propertyName).is(propertyValue));
SkipOperation skipOperation = skip((long) (pageable.getPageNumber() * pageable.getPageSize()));
LimitOperation limitOperation = limit(pageable.getPageSize());
SortOperation sortOperation = sort(pageable.getSort());
return newAggregation(matchOperation, skipOperation, limitOperation, sortOperation);

This definitely worked, but, as I was saying, it feels hacky. Is there a way to get the count for the PageImpl instance without essentially having to run the query twice?

Answer 1

your question has helped me get around the same problem of paging with aggregation and so I did a little digging and came up with a solution to your problem. I know it's a bit late but someone might get use out of this answer. I am in no way a Mongo expert so if what I am doing is bad practice or not very performant please don't hesitate to let me know.

Using group, we can add the root documents to a set and also count.

group().addToSet(Aggregation.ROOT).as("documents")
       .count().as("count"))

Here is my solution for almost the exact same problem you were facing.

private Page<Customer> searchWithFilter(final String filterString, final Pageable pageable, final Sort sort) {
    final CustomerAggregationResult aggregationResult = new CustomerAggregationExecutor()
        .withAggregations(match(new Criteria()
                .orOperator(
                    where("firstName").regex(filterString),
                    where("lastName").regex(filterString))),
            skip((long) (pageable.getPageNumber() * pageable.getPageSize())),
            limit(pageable.getPageSize()),
            sort(sort),
            group()
                .addToSet(Aggregation.ROOT).as("documents")
                .count().as("count"))
        .executeAndGetResult(operations);
    return new PageImpl<>(aggregationResult.getDocuments(), pageable, aggregationResult.getCount());
}

CustomerAggregationResult.java

@Data
public class CustomerAggregationResult {

  private int count;
  private List<Customer> documents;

  public static class PageableAggregationExecutor {

    private Aggregation aggregation;

    public CustomerAggregationExecutor withAggregations(final AggregationOperation... operations) {
      this.aggregation = newAggregation(operations);
      return this;
    }

    @SuppressWarnings("unchecked")
    public CustomerAggregationResult executeAndGetResult(final MongoOperations operations) {
        return operations.aggregate(aggregation, Customer.class, CustomerAggregationResult.class)
            .getUniqueMappedResult();
    }

  }
}

Really hope this helps.

EDIT: I had initially created a generic PageableAggregationResult with List but this returns a IllegalArgumentException as I pass PageableAggregationResult.class with no type for T. If I find a solution for this I will edit this answer as I want to be able to aggregate multiple collections eventually.