Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MongoDB - too much data for sort() with no index error

Tags:

I am using MongoDB 1.6.3, to store a big collection (300k+ records). I added a composite index.

db['collection_name'].getIndexes()
[
    {
        "name" : "_id_",
        "ns" : "db_name.event_logs",
        "key" : {
            "_id" : 1
        }
    },
    {
        "key" : {
            "updated_at.t" : -1,
            "community_id" : 1
        },
        "ns" : "db_name.event_logs",
        "background" : true,
        "name" : "updated_at.t_-1_community_id_1"
    }
]

However, when I try to run this code:

db['collection_name']
  .find({:community_id => 1})
  .sort(['updated_at.t', -1])
  .skip(@skip)
  .limit(@limit)

I am getting:

Mongo::OperationFailure (too much data for sort() with no index. add an index or specify a smaller limit)

What am I doing wrong?

like image 756
Vlad Zloteanu Avatar asked Dec 09 '10 14:12

Vlad Zloteanu


People also ask

Does MongoDB sort use index?

If the sort keys correspond to the index keys or an index prefix, MongoDB can use the index to sort the query results. A prefix of a compound index is a subset that consists of one or more keys at the start of the index key pattern. The following query and sort operations use the index prefixes to sort the results.

Does MongoDB support sorting?

MongoDB can perform sort operations on a single-field index in ascending or descending order. In compound indexes, the sort order determines whether the index can be sorted.

What is ascending index in MongoDB?

For a single-field index and sort operations, the sort order (i.e. ascending or descending) of the index key does not matter because MongoDB can traverse the index in either direction. The value of index is the type of index. For example, 1 indicates ascending order and -1 specifies the descending order.

What sorting algorithm does MongoDB use?

If MongoDB cannot obtain the sort order via an index scan, then MongoDB uses a top-k sort algorithm. This algorithm buffers the first k results (or last, depending on the sort order) seen so far by the underlying index or collection access.


1 Answers

Try adding {community_id: 1, 'updated_at.t': -1} index. It needs to search by community_id first and then sort.

like image 62
pingw33n Avatar answered Oct 07 '22 15:10

pingw33n