Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

skip a mongo capped collection

I have a very large capped collection in mongodb. Given that the capped collection structure is predictable (i.e. sort is predefined, memory footprint is predefined, etc), is there a better way to get a cursor on the LATEST item inserted instead of iterating?

In other words, what I'm doing right now is to get the size of my collection (n), and then create a cursor that sets skip=n-1 to put me at the end of the collection. Then I iterate on the cursor and handle all new additions to the collection.

The problem with this approach is that my collection is huge. lets say 11m records. that takes 20 minutes to skip. Which means that when my cursor starts emitting data, its 20 minutes behind.

like image 369
Oren Mazor Avatar asked Nov 05 '22 02:11

Oren Mazor


2 Answers

Try db.cappedCollection.find().limit(1).sort({$natural:-1}) .

like image 154
Tad Marshall Avatar answered Nov 29 '22 05:11

Tad Marshall


Have you tried indexing the collection and using $gt - this should be faster although the index will have some impact on the speed of the writes to the collection.

like image 40
geakie Avatar answered Nov 29 '22 07:11

geakie