Where does FindAndModify fit into CQRS?

Question

In my current application, I have the following MongoDB FindAndModify call to the MongoDB server

public static TEntity PeekForRemoveSync<TEntity>(this MongoCollection<TEntity> collection, string reason)
    where TEntity : class, IEntity, IUpdatesTrackable, ISyncable, IDeletable, IApprovable
{
    if (reason == null)
    {
        throw new ArgumentNullException(nameof(reason));
    }

    IMongoQuery query = Query.And(
        Query<TEntity>.EQ(e => e.SyncInfo.IsDirty, true),
        Query<TEntity>.NE(e => e.Deletion, null),
        Query<TEntity>.NE(e => e.Approval, null),
        Query<TEntity>.EQ(e => e.SyncInfo.HumanInvestigateFlag, null),
        Query.Or(
            Query<TEntity>.EQ(e => e.SyncInfo.IsUpdateInProgress, false),
            Query<TEntity>.LT(e => e.SyncInfo.UpdateStartInfo.OccuredOn, DateTime.UtcNow.AddSeconds(-SyncConstants.PeekExpireTimeInSeconds))
        ),
        Query<TEntity>.LTE(e => e.SyncInfo.PeekedCount, MaxPeekedCount)
    );

    return PeekForSync(collection, query, reason);
}

private static TEntity PeekForSync<TEntity>(this MongoCollection<TEntity> collection, IMongoQuery query, string reason)
    where TEntity : class, IEntity, IUpdatesTrackable, ISyncable, IDeletable
{
    UpdateBuilder<TEntity> update = Update<TEntity>
        .Inc(e => e.SyncInfo.PeekedCount, 1)
        .Set(e => e.SyncInfo.UpdateStartInfo, new OccuranceWithReason(reason))
        .Set(e => e.SyncInfo.IsUpdateInProgress, true);

    SortByBuilder<TEntity> sort = SortBy<TEntity>.Descending(e => e.LastUpdatedOn);

    FindAndModifyArgs fmArgs = new FindAndModifyArgs
    {
        Query = query,
        Update = update,
        SortBy = sort,
        VersionReturned = FindAndModifyDocumentVersion.Modified
    };

    FindAndModifyResult result = collection.FindAndModify(fmArgs);

    return result.ModifiedDocument != null
        ? result.GetModifiedDocumentAs<TEntity>()
        : null;
}

Am I reinventing yet another queue system with MongoDB? That's not my intention here but it's what it's. Just ignore the business logic there.

In CQRS, Query and Command mean the following (I believe):

Command: Changes the state of the system and return no values.

Query: Doesn't change the state of the system and return some values.

If that's the case, where does my above FindAndModify call (or any other similar one like this query) fit?

Answer 1

This is going to be opinionated, but here is my take on it:

A CQRS Command should usually take some specific action on a single aggregate entity.

Examples of commands:

• ChangePersonName
• PlaceOrder
• DeleteAccount

If data is needed in the implementation of a command handler implementation, it should come from a document that is found strictly by its document ID. (Or ObjectId in Mongo)

In some document databases, querying by something other than the ID is an eventually consistent operation. If you rely on the results of a query to decide what to act on, then you may be missing some items when an index is stale. How would you know that the command had truly completed against all data? You wouldn't.

If the query results were indeed fully consistent (which it may be in Mongo, I'm not 100% certain), even then you have the problem of timing. You may have intended to act on all data that matched your query, but perhaps a few milliseconds later new data gets inserted that would have matched the query. Since the scope of the operation wasn't constrained to a specific document (or set of documents), you'd really have no way to know if you were truly done or not.

In general, I'd say that executing a query (on anything other than the document ID) from within a CQRS command is not a good idea.

FindAndModify looks like it is used for patching. You may indeed need to do that occasionally, but it wouldn't fall in line with CQRS patterns. I would reserve it for maintenance tasks - outside of the main CQRS part of the system.

Answer 2

I'm not sure I got your question right, so here's the answer to what I understood:

I like to see CQRS as a combination of a concurrency pattern and a data model guideline.

The data model guideline is the more obvious one, after all, you're now using more than one model. In a non-CQRS-scenario, one might have the following object for an (unfinished) order or cart:

{  items : [ { 'sku' : '235423', 'qty' : 3, 'price' : 4.34, ... }, ... ],
   orderStatus : 'in_cart',
   customerId : null,
   ...
}

Now, updating that model in itself would be easy, but CQRS tells you not to. Suppose I want to add product '1234' to that cart. I could post back the entire new cart, sure. But that is both inefficient (all I need is the info 'i want to add 1 product 1234 to cart X'), and irritating, because the total price for instance must be ignored by the server anyway.

At its heart, however, I CQRS is mostly a concurrency scheme - at least in combination with Event Sourcing. By keeping records of who changed what when, we can centralize the code that is responsible for applying the changes, for handling conflicts, allowing undo/redo and keeping an audit log, all at the same time. Of course, that comes at the expense of eventual consistency, but the notion is that an object that has many concurrent writers doesn't have a well-defined 'current' state anyway (that's how I interpret it, at least).

Also, looking at that cart again, maybe updating the model isn't trivial after all, because the cart probably represents a lock on the objects in the cart (at least for sites with high likelihood of conflicts, e.g. flash sales - if the product's in the cart, I'd like to know that I can actually buy it, say for 10 minutes). Without command objects, would have to analyze how the document transitioned from state 1 to state 2 and what changed, exactly, as to not release any existing locks, but maybe acquire a new one (or release one and acquire a new one).

Instead, you could send an AddToCartCommand:

{ item : { 'sku' : '4242342', 'qty' : 4 },
  cartId : 53543 }

If that product was sold out too quickly, it doesn't hurt, since we're not abandoning the basket - the add to cart command failed, but the state of the cart remained completely unchanged. If the site is under high load and the buyer added yet another product, that command might succeed. With traditional request/response patterns, that would not have been easy, if possible at all.

This is probably not the best example, but suppose the cart was filled by multiple users concurrently, then the simple 'last write wins' concurrency (no lock) or an optimistic look (error: someone else changed the document in between) would be problematic.

In CQRS, since we're not doing immediate updates, we can steer clear of both RPC (a 'method call' that tells me what happened immediately) and overly client-specific interfaces. Also, unlike RPC vs. REST, CQRS goes at least one layer deeper. While it certainly affects the client code a lot, it has tremendous implications on the way the server processes information, in an asynchronous, potentially distributed manner that might even involve completely different data stores (since the models are different anyway).

In any case, I think the code you posted might be the point where the two models communicate, i.e. where the changes must be applied, which can be tricky. For one thing, that requires (distributed) transaction, hence a lot of locking going on and the locking requires atomic updates, so the findAndModify appears to be a necessity in this case.

I'm not sure what the peeking does, exactly, but to me your code looks like it's trying to get a short lock on the document (from the IsDirty : true and IsUpdateInProgress : false criteria).

In a word, I'd say CQRS is the async of the web.