Why the queries inside the system.profile collection are being overwritten

Question

I am trying to collect all my queries happening on to mongodb with respect to my Applciation . So i ahve set up profiling level to 2 so that it will log all the queries . As part of the architecture , as soon as the user logs in there will be contonious updates happening on to mongodb .

But i observed is that the system.profile count is being reduced don't know why

> db.system.profile.count()
322
> db.system.profile.count()
351
> db.system.profile.count()
202
> db.system.profile.count()
136
> db.system.profile.count()
233

Why in my case the queries are being overwritten ??

Is there any possibility that i can record all my distinict queries happening for my Application.

I need this because i can remove / add some of the indexes on one of my collection .

Answer 1

system.profile is a capped collection with default size of 1MB so documents are purged on natural (with respect to insertion timestamp) basis in order to make room for newer documents.

In order to increase the size of system.profile collection (say 10 MB) do:

1) Stop profiler : db.setProfilingLevel(0)

2) Drop system.profile: db.system.profile.drop()

3) Create a much larger system.profile collection manually: db.createCollection( "system.profile", { capped: true, size: 1024 * 1024 * 10 } )

4) Enable profiling again

If, on average, a 1MB collection fitted around 200-300 documents, the new settings will fit approximately 2,000 - 3,000 documents. Increase the system.profile size as needed.

Answer 2

Here's the official documentation link https://docs.mongodb.org/manual/tutorial/manage-the-database-profiler/#profiler-overhead

You need to make the collection change size of system.profile Collection