MongoDB hosted in Azure Cosmos DB: Sharding vs partitioning

Question

We want to use MongoDB for our database, and we want to use the MongoDB API to avoid to be "locked in" to Azure Cosmos DB hosting.

We use .Net Core and the MongoDB.Driver package (to be able to easily switch between on-prem, Atlas, Azure Cosmos hsoting etc.) to communicate with the MongoDB instance, so far so good.

To be able to handle future growth of data volumes (size and performance) I want to have my collections sharded. As I understand the strategy Cosmos DB use is partitioning with partition keys, but since we use the MongoDB.Driver I can not find anyway to specify partitionkeys in my queries.

"Plain" MongoDB use sharding instead, and you can set up a document property that should be used as a delimiter for how your data should be sharded.

So, my guess is that sharding is the way to go (since partionkeys is a Cosmos feature) but I can not manage to get it working.

The "MongoDB shell" in Azure Portal do not understand the sh.shardCollection command and if I connect with a MongoDB shell from my client I get the following error:

globaldb:PRIMARY> use sampledatabase
switched to db sampledatabase
globaldb:PRIMARY> sh.shardCollection("sampledatabase.Event", { TenantId: 1 } )
2018-06-21T12:03:06.522+0200 E QUERY    [thread1] Error: not connected to a mongos :

How do I proceed to get sharding up and running in a MongoDB instance hosted in Azure Cosmos?

Answer 1

The CosmosDB Mongo api endpoint exposes a MongoD interface with replica-set enabled instead of a MongoS interface. Hence, you will need to use db.runCommand instead of the "sh" sharding commands to create a sharded collection.

You can find more details at https://docs.microsoft.com/en-us/azure/cosmos-db/partition-data#mongodb-api

Answer 2

I later found out that you can create sharded collections with the Microsoft.Azure.Documents.Client.

You must use the funky syntax @"/'$v'/ShardingKey/'$v'" for it to work. Then you can use a property on your document named ShardingKey that will blend well with MongoDB.Driver library.

                        _client.CreateDocumentCollectionAsync(databaseUri,
                        new DocumentCollection
                        {
                            Id = documentCollection.Id,
                            PartitionKey =
                                new PartitionKeyDefinition
                                {
                                    Paths = new Collection<string> {@"/'$v'/ShardingKey/'$v'"}
                                }
                        }, new RequestOptions {OfferThroughput = 1100}).Wait();

See https://blog.olandese.nl/2017/12/13/create-a-sharded-mongodb-in-azure-cosmos-db/ for reference