Unable to get a covered query to use index only in mongodb

Question

I am trying to use a covering index to implement stemming text search on my app which uses mongodb.

I've got the following index set:

ensureIndex({st: 1, n: 1, _id: 1});

But when I run explain() on my query, I can never get the indexOnly to read true, no matter what I do.

db.merchants.find({st: "Blue"}, {n:1,_id:1}).explain()
{
    "cursor" : "BtreeCursor st_1_n_1__id_1",
    "nscanned" : 8,
    "nscannedObjects" : 8,
    "n" : 8,
    "millis" : 0,
    "nYields" : 0,
    "nChunkSkips" : 0,
    "isMultiKey" : true,
    "indexOnly" : false,
    "indexBounds" : {
        "st" : [
            [
                "Blue",
                "Blue"
            ]
        ],
        "n" : [
            [
                {
                    "$minElement" : 1
                },
                {
                    "$maxElement" : 1
                }
            ]
        ],
        "_id" : [
            [
                {
                    "$minElement" : 1
                },
                {
                    "$maxElement" : 1
                }
            ]
        ]
    }
}

I've already figured out that the ordering of the keys in the index matter somehow. For instance if I used {_id, n:1, st:1} it wasn't using this index at all to perform the query. I also read somewhere that too few documents could trigger unpredictable behaviour with explain() since multiple strategies are equally fast. But in this case, I see that its using the right index, but its not using just the index. What is this happening?

I am using mongoid, and mongo 2.0.8 I believe.

UPDATE:

Switched over to using Mongoid v3.1.4 and mongod v2.2

Here is the query that mongod is seeing from mongoid: Mon Jul 15 10:47:26 [conn14] runQuery called spl_development.merchants { $query: { st: { $regex: "cr", $options: "i" } }, $explain: true } Mon Jul 15 10:47:26 [conn14] query spl_development.merchants query: { $query: { st: { $regex: "cr", $options: "i" } }, $explain: true } ntoreturn:0 keyUpdates:0 locks(micros) r:212 nreturned:1 reslen:393 0ms

So the projection isn't being sent to the mongod layer and only just handles it in the application layer. Not ideal!

This has been recognized as a bug in mongoid and can be tracked here: https://github.com/mongoid/mongoid/issues/3142

Answer 1

I expect your query cannot use a covered index because you have a field with an array included in the index. This is suggested in the explain with "isMultiKey" : true.

As noted in the documentation (Create Indexes that Support Covered Queries):

MongoDB cannot use a covered query if any of the indexed fields in any of the documents in the collection includes an array. If an indexed field is an array, the index becomes a multi-key index and cannot support a covered query.

Answer 2

I wasn't able to reproduce the problem in 2.2.2, but add .sort({n: 1, _id: 1}) into the chain. Because you're not sorting, you're asking for the docs in whatever find order mongo wishes to use, and if that doesn't match the order in the index ($natural order, for instance) it still has to read the docs.

db.merchants.find({st: "Blue"}, {n:1,_id:1}).sort({n: 1, _id: 1}).explain()