CouchDB Filtered Replication

Question

Trying out filters for replication, I stumbled upon a problem. While my filter is working as an entry in the _replicator database, I doesn't when using cURL.

The filter in the design document is:

{
    "_id": "_design/partial",
    "filters": {
        "mobile": "function(doc, req) { 
            if (doc._attachments) {
                var result = new Boolean(true);
                for (attachment in doc._attachments) {
                    if (attachment.content_type == 'image/jpeg') {
                        return true;
                    }
                    if (doc._attachments.length > 1024) {
                        result = false; 
                    }
                }
                return result;
            } else {
                return true;
            }
        }"
    }
}

The cURL line:

curl -X POST http://admin:[email protected]:5985/_replicate -d '{\"source\":\"http://admin:[email protected]:5984/docs2\",\"target\":\"docs2_partial\",\"filter\":\"partial/mobile\",\"create_target\":true}' -H "Content-Type: application/json"

I created _design/partial document on both target and source, but all documents are being replicated. Even the one with an attached binary bigger than 1 MB. Any help is appreciated!

The cURL reply is:

{"ok":true,"session_id":"833ff96d21278a24532d116f57c45f31","source_last_seq":32,"replication_id_version":2,"history":[{"session_id":"833ff96d21278a24532d116f57c45f31","start_time":"Wed, 17 Aug 2011 21:43:46 GMT","end_time":"Wed, 17 Aug 2011 21:44:22 GMT","start_last_seq":0,"end_last_seq":32,"recorded_seq":32,"missing_checked":0,"missing_found":28,"docs_read":28,"docs_written":28,"doc_write_failures":0}]}

Using either " instead of \" or " instead of ' the result is:

{"error":"bad_request","reason":"invalid UTF-8 JSON: [...]}

Answer 1

Now I think perhaps the logic of your filter function simply has a bug. Here is how I read your filter policy:

• All docs that have no attachments pass
• All docs that have an image/jpeg attachment pass
• Docs with more than 1,024 attachments fail
• In any other case, the docs pass

That sounds like perhaps an incorrect policy. Another way to restate this policy is "Docs with more than 1024 attachments fail, everything else passes." However since you wrote so much code, I suspect my summary is not the true policy.

Another quick note, on what looks like a bug. Given:

for (attachment in doc._attachments) { /* ... */ }

The attachment variable will be things like "index.html" or "me.jpeg", i.e. filenames. To get the attachment content-type, you need:

var type;

// This is WRONG
type = attachment.content_type; // type set to undefined

// This is RIGHT
type = doc._attachments[attachment].content_type; // type set to "text/html" etc.

To avoid this bug, you could change your code to make things more clear:

for (attachment_filename in doc._attachments) { /* ... */ }

Next, doc._attachments.length will tell you the number of attachments in the document, not for example the length of the current attachment. It is odd that you test for that inside the loop, because the expression will never change. Are you trying to test for attachment size instead?