Azure CosmosDB I cannot get more than 1500 RUs

Question

I have an application that requires large RUs, but for some reason I cannot get the client app to handle more than 1000-1500 RUs, although the collection is set to 10000 RUs. Obviously I can add more clients, but I need one client to give me at least 10000 RUs then scale that. My requests are simple

    var query = connection.CreateDocumentQuery<DocumentDBProfile>(
    CollectionUri, //cached
    "SELECT * FROM Col1 WHERE Col1.key = '" + partitionKey + "' AND Col1.id ='" + id + "'",
    new FeedOptions
    {
        MaxItemCount = -1,
        MaxDegreeOfParallelism = 10000000,
        MaxBufferedItemCount = 1000,
    }).AsDocumentQuery();
    var dataset = await query.ExecuteNextAsync().ConfigureAwait(false);

The query above is hitting 150,000 partitions, each is within own task (awaiting all at the end) and the client is initialized with TCP and direct mode:

                            var policy = new ConnectionPolicy
                            {
                                EnableEndpointDiscovery = false,
                                ConnectionMode = ConnectionMode.Direct,
                                ConnectionProtocol = Protocol.Tcp,
                            };

The CPU on the client appears to max out, mostly to service the call query.ExecuteNextAsync()

Am I doing anything wrong? Any optimization tips? Is there a lower level API I can use? Is there a way to pre-parse queries or make Json parsing more optimal?

UPDATE I was able to get up to 3000-4000 RU on one client by lowering the number of concurrent requests, and stripping down my deserialized class to one with a single property (id), but I am still 10% of the limit of 50,000 RUs mentioned in the performance guidelines. Not sure what else I could do. Is there any security checks or overhead I can disable in the .Net SDK?

UPDATE2 All our tests are run on Azure in the same region D11_V2. Running multiple clients scales well, so we are client bound not server bound. Still not able to achieve 10% of the performance outlined in the CosmosDB performance guideline

Answer 1

By default the SDK will use a retry policy to mask throttling errors. Have you looked at the RU metrics available on Azure portal to confirm if you are being throttled or not? For more details on this, see tutorial here.

Not sure why the REST API would perform better than the .NET SDK. Can you give some more details on the operation you used here?

The example query you provided is querying a single document with a known partitionkey and id per request. For this kind of point-read operation, it would be better to use DocumentClient.ReadDocumentAsnyc, as it should be cheaper than a query.

Answer 2

It sounds like your sole purpose has become to disprove the documentation of Microsoft. Don't overrate this "50.000 RU/S" value for how you should scale your clients.

I don't think you can get a faster & lower level API than using .NET SDK with TCP & direct mode. The critical part is to use the TCP protocol (which you are). Only Java SDK also has direct mode, i doubt its faster. Maybe .NET Core...

How can your requirement be to "have large RU/s"? That is equivalent to "the application should require us to pay X$ for CosmosDB every month". The requirement should rather be "needs to complete X queries per second" or something like this. You then go on from there. See also the request unit calculator.

A request unit is the cost of your transaction. It depends on how large your documents are, how your collection is configured and on what your are doing. Inserting documents is usually much more expensive than retrieving data. Retrieving data across partitions within one query is more expensive than touching only a single one. A rule of thumb is that writing data is about 5 times more expensive than reading it. I suggest you read the documentation about request units.

The problem with the performance tip of Microsoft is that they don't mention anything about which request should incur those RU/s. I would not expect it to mean: "The most basic request possible will not max out the CPU on the client system if you are still below 50.000 RU/s". Inserting data will get you to those numbers much more easily. I did a very quick test on my local machine, and got the official benchmarking sample up to about 7-8k RU/s using TCP+direct. I did not do anything apart from downloading the code and running it from Visual Studio. So my guess would be that the tips are also about inserting, since the performance testing examples are as well. The example achieves 100.000RU/s incidentally.

There are some good samples from Azure about "Benchmarking" and "Request Units". They should also be good sources for further experiments.

Only one actual tip on how to improve your query: Maybe ditch deserialization to your class, using CreateDocumentQuery(..) or CreateDocumentQuery<dynamic>. Could help your CPU. My first guess would be that your CPU is doing a bunch of that.

Hope this helps in any way.