I have a MongoDB database with 10-12k documents in a collection and I'm experiencing really slow queries when attempting to fetch all documents, like this:
Sales.find()
.where('author').equals(author)
.where('date').gt(startDate.unix()).lt(endDate.unix())
.exec(function(err, results) {
callback();
});
This query fetches around 10.5k documents and it takes 1000-1300ms to execute. I tried removing the "where" conditions - it only makes it slower (more documents fetched?).
Does the problem come from Mongoose, MongoDB, JavaScript or Node? I used to run PHP/MySQL database and it was 10-20 times faster in similar conditions, like fetching 10k+ rows of data. What am I doing wrong?
EDIT
Sales schema:
var salesSchema = new Schema({
author: String,
kind: String,
productID: String,
description: String,
date: String,
amount: String,
transactionID: {
type: String,
unique : true
}
});
Query result from the RoboMongo desktop client:
db.getCollection('sales').find({}).explain()
executionTimeMillis: 46
nReturned: 10359
The problem came from Mongoose. By default, find() will return documents as Mongoose Documents, which costs a lot. By adding lean() to the query, documents are returned as plain JavaScript objects and for this case of 10k+ returned documents, the query time got reduced 3-5 times.
Sales.find()
.where('author').equals(author)
.where('date').gt(startDate.unix()).lt(endDate.unix())
.lean()
.exec(function(err, results) {
callback();
});
Read more here: http://www.tothenew.com/blog/high-performance-find-query-using-lean-in-mongoose-2/
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With