Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

querying on 10 million mongodb documents

Tags:

php

mongodb

I am storing book meta-data like name,authors,price,publisher,etc in a mongodb document. I have about 10 million of these documents and they all are in one collection. The average document size is 1.9 KB. Now i have indexes on name,authors and price. In fact i have 2 indexes on price one in ascending order and one descending order. My mongodb version is 2.2.0 and i am using the php driver to query mongo. The driver's version is 1.12. But when i do a range query on price i get a MongoCursorTimeoutException. In my query i am trying to find books in a certain price range like "price less than 1000 and more than 500".

Increasing the timeout doesn't seem to be a good idea(It is already 30 sec). Is there anything else that i can do to speed up the query process.

EDIT Actually my price index is compound. I have a status field which has an integer value so my price index looks like {price:-1,status:1} and {price:1,status:1} Also I am trying to retrieve 20 documents at a time with PHP.

like image 528
lovesh Avatar asked Sep 18 '12 18:09

lovesh


People also ask

Is MongoDB good for millions of records?

Working with MongoDB and ElasticSearch is an accurate decision to process millions of records in real-time. These structures and concepts could be applied to larger datasets and will work extremely well too.

What is the limit of in query MongoDB?

In MongoDB, the limit() method limits the number of records or documents that you want. It basically defines the max limit of records/documents that you want. Or in other words, this method uses on cursor to specify the maximum number of documents/ records the cursor will return.

How many documents can MongoDB handle?

MongoDB supports no more than 100 levels of nesting for BSON documents. Each object or array adds a level.


1 Answers

As @JohnyHK said my RAM was too low. So increased it to 12 GB and it works now. Thanks everyone for their comments and answers

like image 197
lovesh Avatar answered Oct 08 '22 16:10

lovesh