Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Guaranteeing sort order with an IAsyncCursor query?

I'm using a collection with millions of records populated asynchronously, so there is no guarantee of order.

When querying the MongoDB FindAsync method will allow a filter to be added, but not a sort.

How do I ensure the order of records returned using IAsyncCursor? I have an ascending index on a epoch date-stamp field in my collection, is that enough to guarantee the sort order?.

====================

An IAsyncCursor is returned by the FindAsync method. The records are returned in arbitrary batches which are processed using the following using using/while construct.

var collection = _database.GetCollection<BsonDocument>("restaurants");
var filter = new BsonDocument();
var count = 0;
using (var cursor = await collection.FindAsync(filter))
{
    while (await cursor.MoveNextAsync())
    {
        var batch = cursor.Current;
        foreach (var document in batch)
        {
            // What order will these records be in? how do I guarantee order ascending by epochtimestamp?;
        }
    }
}

Will each async "batch" be sorted with respect to the entirety of the query? How do I guarantee records will be in the correct order?.

The ascending index on the epocdatestamp field would allow me to return a full sorted list(Takes a very long time because of the number of records).

Should I one time rewrite that full sorted list back to the DB in the order I need to guarantee? So that future IAsyncCursor querys will return records in order.

Do I need to do that? or is the ascending index enough?

like image 428
Kickaha Avatar asked Aug 07 '15 14:08

Kickaha


1 Answers

A sort can be added to a "FindOptions" object and given to FindAsync as an argument. In this case it's an ascending sort on a field called epochtimestamp.

Simple when you know how,

    var collection = _database.GetCollection<BsonDocument>("restaurants");
    var filter = new BsonDocument();
    var sort = Builders<BsonDocument>.Sort.Ascending("epochdatestamp");
    var options = new FindOptions<BsonDocument>
                  {
                    Sort = sort
                  };

    var count = 0;
    using (var cursor = await collection.FindAsync(filter, options))
    {
        while (await cursor.MoveNextAsync())
        {
            var batch = cursor.Current;
            foreach (var document in batch)
            {
                // process document
                count++;
            }
        }
    }
like image 166
Kickaha Avatar answered Oct 20 '22 03:10

Kickaha