Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

In spring data mongodb how to achieve pagination for aggregation

In spring data mongodb using mongotemplate or mongorepository, how to achieve pagination for aggregateion

like image 677
goutham Avatar asked Dec 23 '15 01:12

goutham


People also ask

Can we use $and in aggregate MongoDB?

You can use $and with aggregation but you don't have to write it, and is implicit using different filters, in fact you can pipe those filters in case one of them needs a different solution. $match takes { <query> } . That means, it can take $and just fine.

What is pagination in MongoDB?

In MongoDB, the pagination phenomenon is used to get an output that can be easily understandable. Pagination is a procedure to display large disarranged output in a page format. With the help of pagination, the result can be retrieved faster as compared to the general methods of MongoDB.

Is MongoDB good at aggregation?

MongoDB Aggregation goes further though and can also perform relational-like joins, reshape documents, create new and update existing collections, and so on. While there are other methods of obtaining aggregate data in MongoDB, the aggregation framework is the recommended approach for most work.


2 Answers

Here is my generic solution:

public Page<ResultObject> list(Pageable pageable) {
    // build your main stages
    List<AggregationOperation> mainStages = Arrays.asList(match(....), group(....));
    return pageAggregation(pageable, mainStages, "target-collection", ResultObject.class);
}

public <T> Page<T> pageAggregation(
        final Pageable pageable,
        final List<AggregationOperation> mainStages,
        final String collection,
        final Class<T> clazz) {
    final List<AggregationOperation> stagesWithCount = new ArrayList<>(mainStages);
    stagesWithCount.add(count().as("count"));
    final Aggregation countAgg = newAggregation(stagesWithCount);
    final Long count = Optional
            .ofNullable(mongoTemplate.aggregate(countAgg, collection, Document.class).getUniqueMappedResult())
            .map(doc -> ((Integer) doc.get("count")).longValue())
            .orElse(0L);

    final List<AggregationOperation> stagesWithPaging = new ArrayList<>(mainStages);
    stagesWithPaging.add(sort(pageable.getSort()));
    stagesWithPaging.add(skip(pageable.getOffset()));
    stagesWithPaging.add(limit(pageable.getPageSize()));
    final Aggregation resultAgg = newAggregation(stagesWithPaging);
    final List<T> result = mongoTemplate.aggregate(resultAgg, collection, clazz).getMappedResults();

    return new PageImpl<>(result, pageable, count);
}
like image 105
nyxz Avatar answered Sep 17 '22 04:09

nyxz


This is an answer to an old post, but I'll provide an answer in case anyone else comes along while searching for something like this.

Building on the previous solution by Fırat KÜÇÜK, giving the results.size() as the value for the "total" field in the PageImpl constructor will not making paging work the way, well, you expect paging to work. It sets the total size to the page size every time, so instead, you need to find out the actual total number of results that your query would return:

public Page<UserListItemView> list(final Pageable pageable) {
    long total = getCount(<your property name>, <your property value>);

    final Aggregation agg = newAggregation(
        skip(pageable.getPageNumber() * pageable.getPageSize()),
        limit(pageable.getPageSize())
    );

    final List<UserListItemView> results = mongoTemplate
        .aggregate(agg, User.class, UserListItemView.class)
        .getMappedResults();

    return new PageImpl<>(results, pageable, total);
}

Now, then, the best way to get the total number of results is another question, and it is one that I am currently trying to figure out. The method that I tried (and it worked) was to almost run the same aggregation twice, (once to get the total count, and again to get the actual results for paging) but using only the MatchOperation followed by a GroupOperation to get the count:

private long getCount(String propertyName, String propertyValue) {
    MatchOperation matchOperation = match(Criteria.where(propertyName).is(propertyValue));
    GroupOperation groupOperation = group(propertyName).count().as("count");
    Aggregation aggregation = newAggregation(matchOperation, groupOperation);
    return mongoTemplate.aggregate(aggregation, Foo.class, NumberOfResults.class).getMappedResults().get(0).getCount();
}

private class NumberOfResults {
    private int count;

    public int getCount() {
        return count;
    }

    public void setCount(int count) {
        this.count = count;
    }
}

It seems kind of inefficient to run nearly the same query twice, but if you are going to page results, the pageable object must know the total number of results if you really want it to behave like paging. If anyone can improve on my method to get the total count of results, that would be awesome!

Edit: This will also provide the count, and it is simpler because you do not need a wrapper object to hold the result, so you can replace the entire previous code block with this one:

private long getCount(String propertyName, String propertyValue) {
    Query countQuery = new Query(Criteria.where(propertyName).is(propertyValue));
    return mongoTemplate.count(countQuery, Foo.class);
}
like image 41
Steve Storck Avatar answered Sep 19 '22 04:09

Steve Storck