Change notification in CouchDB when a field is set

Question

I'm trying to get notifications in a CouchDB change poll as soon as pre-defined field is set or changed. I've already had a look at filters that can be used for filtering change events(db/_changes?filter=myfilter). However, I've not yet found a way to include this temporal information, because you can only get the current version of the document in this filter functions.

Is there any possibility to create such a filter?

If it does not work, I could export my field to a separate database and the only poll for changes in that db, but I'd prefer to keep together my data for obvious reasons.

Thanks in advance!

Answer 1

You are correct: filters and _changes feeds can only see snapshots of a document. What you need is a function which can see the old document and the new document and act correctly. But that is unavailable in _filters and _changes.

Obviously your client code knows if it updates that field. You might update your client code however there is a better solution.

Update functions can access both documents. I suggest you make an _update function which notices the field change and flags that in the document. Next you have a simple filter checking for that flag. The best part is, you can use a rewrite function to make the HTTP API exactly the same as before.

1. Create an update function to flag interesting updates

Your _design/myapp would be {"updates", "smart_updater": "(see below)"}. Update functions are very flexible (see my recent update handlers walkthrough). However we only want to mimic the normal HTTP/JSON API.

Your updates.smart_updater field would look like this:

function (doc, req) {
    var INTERESTING = 'dollars'; // Set me to the interesting field.

    var newDoc = JSON.parse(req.body);
    if(newDoc.hasOwnProperty(INTERESTING)) {
        // dollars was set (which includes 0, false, null, undefined
        // values. You might test for newDoc[INTERESTING] if those
        // values should not trigger this code.
        if((doc === null) || (doc[INTERESTING] !== newDoc[INTERESTING])) {
            // The field changed or created!
            newDoc.i_was_changed = true;
        }
    }

    if(!newDoc._id) {
        // A UUID generator would be better here.
        newDoc._id = req.id || Math.random().toString();
    }

    // Return the same JSON the vanilla Couch API does.
    return [newDoc, {json: {'id': newDoc._id}}];
}

Now you can PUT or POST to /db/_design/myapp/_update/[doc_id] and it will feel just like the normal API except if you update the dollars field, it will add an additional flag, i_was_changed. That is how you will find this change later.

2. Filter for documents with the changed field

This is very straightforward:

function(doc, req) {
    return doc.i_was_changed;
}

Now you can query the _changes feed with a ?filter= parameter. (Replication also supports this filter, so you could pull to your local system all documents which most recently changed/created the field.

That is the basic idea. The remaining steps will make your life easier if you already have lots of client code and do not want to change the URLs.

3. Use rewriting to keep the HTTP API the same

This is available in CouchDB 0.11, and the best resource is Jan's blog post, nice URLs in CouchDB.

Briefly, you want a vhost which sends all traffic to your rewriter (which itself is a flexible "bouncer" to all design doc functionality based on the URL).

curl -X PUT http://example.com:5984/_config/vhosts/example.com \
  -d '"/db/_design/myapp/_rewrite"'

Then you want a rewrites field in your design doc, something like (not tested)

[
    {
        "comment": "Updates should go through the update function",
        "method": "PUT",
        "from": "db/*",
        "to"  : "db/_design/myapp/_update/*"
    },
    {
        "comment": "Creates should go through the update function",
        "method": "POST",
        "from": "db/*",
        "to"  : "db/_design/myapp/_update/*"
    },
    {
        "comment": "Everything else is just like normal",
        "from": "*",
        "to"  : "../../../*"
    }
]

(Once again, I got this code from examples and existing code I have laying around but it's not 100% debugged. However I think it makes the idea very clear. Also remember this step is optional however the advantage is, you never have to change your client code.)