Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Cosmos DB with multiple partition keys

We're looking at potentially using a single Cosmos DB collection to hold multiple document types in a multi-tenanted environment using a tenant ID as the partition key. The path to tenant id may change in each document type and I am therefore looking at various was of exposing the partition key to Cosmos DB to enable correct partitioning / querying.

I have noticed that the Paths property of DocumentCollection.PartitionKey is a collection and was therefore wondering whether it is possible to pass multiple paths during the creation of a document collection and what the behaviour of this might be. Ideally, I would like Cosmos to scan each of these paths and use the first value or aggregate of values as the partition key but cannot find any documentation suggesting that this is indeed the behaviour.

The MSDN documentation for this property is pretty useless and none of the associated documentation seems to answer the question. Does anyone know about or previously used multiple partition key paths in a collection?

To be clear, I'm looking for links to additional documentation about and/or direct experience of the Cosmos DB's behaviour when specifying multiple partition keys in the PartitionKey.Paths collection when creating a DocumentCollection.

This question has also been posted in the Azure Community Support forums.

Thanks, Ian

like image 784
ibebbs Avatar asked Nov 16 '17 14:11

ibebbs


People also ask

Can we have multiple partition keys in Cosmos DB?

Multiple partition keys are not supported in Azure Cosmos DB but we can create a synthetic partition key which is the concatenation of multiple properties.

Can you have multiple partition keys?

In contrast to a simple partition key, a composite partition key uses two or more columns to identify where data resides. Composite partition keys are used when the data stored is too large to reside in a single partition. Using more than one column for the partition key breaks the data into chunks, or buckets.

Is partition key unique in Cosmos DB?

The partition key combined with the unique key guarantees the uniqueness of an item within the scope of the container. For example, consider an Azure Cosmos DB container with Email address as the unique key constraint and CompanyID as the partition key.

How many partitions does Cosmos DB have?

There is no limit to the number of logical partitions in your container. Each logical partition can store up to 20GB of data. Good partition key choices have a wide range of possible values.


2 Answers

The best way to do this is to assign a generic partition key like “pk”, then assign this value based on each of your object types. You can for example, manage this during serialization by having different properties for each class to be serialized to “pk”.

The reason partition key is an array in DocumentCollection.PartitionKey is to allow us to introduce compound partition keys, where the combination of multiple properties like (“firstName”, “lastName”) form the partition key. This is a little different from what you need.

like image 185
Aravind Krishna R. Avatar answered Oct 01 '22 04:10

Aravind Krishna R.


Further to the above, I ended up adding a partition key property to the document container as suggested by Aravind and then used David Fowler's excellent QueryInteceptor nuget package to apply an ExpressionVisitor which translated any equivalence expression relating to the specific document type's tenant id property into a equivalence expression on the partition key property. This ensured that queries would be performed against only the single, correct partition. Furthermore, I was able to use the ExpressionVisitor as a safety feature in that it is able to enforce that all queries provide a filter on tenant id (as, obviously, tenants should never be able to see each others documents) and if none has been specified then no records are returned (an invalid equivalence expression is added to the partition key property).

This has been tested and seems to be working well.

like image 36
ibebbs Avatar answered Oct 01 '22 04:10

ibebbs