How does Cosmos DB Continuation Token work?

Question

At first sight, it's clear what the continuation token does in Cosmos DB: attaching it to the next query gives you the next set of results. But what does "next set of results" mean exactly?

Does it mean:

1. the next set of results as if the original query had been executed completely without paging at the time of the very first query (skipping the appropriate number of documents)?
2. the next set of results as if the original query had been executed now (skipping the appropriate number of documents)?
3. Something completely different?

---

Answer 1. would seem preferable but unlikely given that the server would need to store unlimited amounts of state. But Answer 2. is also problematic as it may result in inconsistencies, e.g. the same document may be served multiple times across pages, if the underlying data has changed between the page queries.

Answer 1

Cosmos DB query executions are stateless at the server side. The continuation token is used to recreate the state of the index and track progress of the execution.

"Next set of results" means, the query is executed again on from a "bookmark" from the previous execution. This bookmark is provided by the continuation token.

1. Documents created during continuations

They may or may not be returned depending on the position of insert and query being executed.

Example:

SELECT * FROM c ORDER BY c.someValue ASC

Let us assume the bookmark had someValue = 10, the query engine resumes processing using a continuation token where someValue = 10.

If you were to insert a new document with someValue = 5 in between query executions, it will not show up in the next set of results.

If the new document is inserted in a "page" that is > the bookmark, it will show up in next set of results

2. Documents updated during continuations

Same logic as above applies to updates as well (See #4)

3. Documents deleted during continuations

They will not show up in the next set of results.

4. Chances of duplicates

In case of the below query,

SELECT * FROM c ORDER BY c.remainingInventory ASC

If the remainingInventory was updated after the first set of results and it now satisfies the ORDER BY criteria for the second page, the document will show up again.

---

Cosmos DB doesn’t provide snapshot isolation across query pages. However, as per the product team this is an incredibly uncommon scenario because queries over continuations are very quick and in most cases all query results are returned on the first page.

Answer 2

Based on preliminary experiments, the answer seems to be option #2, or more precisely:

1. Documents created after serving the first page are observable on subsequent pages
2. Documents updated after serving the first page are observable on subsequent pages
3. Documents deleted after serving the first page are omitted on subsequent pages
4. Documents are never served twice

The first statement above contradicts information from MSFT (cf. Kalyan's answer). It would be great to get a more qualified answer from the Cosmos DB Team specifying precisely the semantics of retrieving pages. This may not be very important for displaying data in the UI, but may be essential for data processing in the backend, given that there doesn't seem to be any way of disabling paging when performing a query (cf. Are transactional queries possible in Cosmos DB?).

---

Experimental method

I used Sacha Bruttin's Cosmos DB Explorer to query a collection with 5 documents, because this tool allows playing around with the page size and other request options.

The page size was set to 1, and Cross Partition Queries were enabled. Different queries were tried, e.g. SELECT * FROM c or SELECT * FROM c ORDER BY c.name.

After retrieving page 1, new documents were inserted, and some existing documents (including documents that should appear on subsequent pages) were updated and deleted. Then all subsequent pages were retrieved in sequence.

(A quick look at the source code of the tool confirmed that ResponseContinuationTokenLimitInKb is not set.)