MongoDB FindAndModify Sorting

Question

I am using FindAndModify in MongoDB in several concurrent processes. The collection size is about 3 million entries and everything works like a blast as long as I don't pass a sorting option (by an indexed field). Once I try to do so, the following warning is spawned to the logs:

warning: ClientCursor::yield can't unlock b/c of recursive lock ns: test_db.wengine_queue top: 
{ 
opid: 424210, 
active: true, 
lockType: "write", 
waitingForLock: false, 
secs_running: 0, 
op: "query", 
ns: "test_db", 
query: { 
    findAndModify: "wengine_queue", 
    query: { 
            locked: { $ne: 1 }, 
            rule_completed: { $in: [ "", "0", null ] }, 
            execute_at: { $lt: 1324381363 }, 
            company_id: 23, 
            debug: 0, 
            system_id: "AK/AK1201" 
        }, 
    update: { 
            $set: { locked: 1 } 
        }, 
    sort: { 
            execute_at: -1 
        } 
}, 
client: "127.0.0.1:60873", 
desc: "conn", 
threadId: "0x1541bb000", 
connectionId: 1147, 
numYields: 0 
}

I do have all the keys from the query indexed, here they are:

PRIMARY> db.wengine_queue.getIndexes()
[
{
    "v" : 1,
    "key" : {
        "_id" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "_id_"
},  
{
    "v" : 1,
    "key" : {
        "system_id" : 1,
        "company_id" : 1,
        "locked" : 1,
        "rule_completed" : 1,
        "execute_at" : -1,
        "debug" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "system_id_1_company_id_1_locked_1_rule_completed_1_execute_at_-1_debug_1"
},  
{
    "v" : 1,
    "key" : {
        "debug" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "debug_1"
},
{
    "v" : 1,
    "key" : {
        "system_id" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "system_id_1"
},
{
    "v" : 1,
    "key" : {
        "company_id" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "company_id_1"
},
{
    "v" : 1,
    "key" : {
        "locked" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "locked_1"
},
{
    "v" : 1,
    "key" : {
        "rule_completed" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "rule_completed_1"
},
{
    "v" : 1,
    "key" : {
        "execute_at" : -1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "execute_at_-1"
},
{
    "v" : 1,
    "key" : {
        "thread_id" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "thread_id_1"
},
{
    "v" : 1,
    "key" : {
        "rule_id" : 1
    },
    "ns" : "test_db.wengine_queue",
    "name" : "rule_id_1"
}
]

Is there any way around this?

Answer 1

For those interested -- I had to create a separate index ending with the key that the set is to be sorted by.

Answer 2

That warning is thrown when an operation that wants to yield (such as long updates, removes, etc.) cannot do so because it cannot release the lock it's holding for whatever reason.

Do you have the field you're sorting on indexed? If not adding an index for that will probably remove the warnings.