Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

limits of number of collections in databases

Can anyone say are there any practical limits for the number of collections in mongodb? They write here https://docs.mongodb.com/manual/core/data-model-operations/#large-number-of-collections:

Generally, having a large number of collections has no significant performance penalty, and results in very good performance.

But for some reason mongodb set limit 24000 for the number of namespaces in the database, it looks like it can be increased, but I wonder why it has some the limit in default configuration if having many collections in the database doesn't cause any performance penalty?

Does it mean that it's a viable solution to have a practically unlimited number of collections in one database, for example, to have one collection of data of one account in a database for the multitenant application, having, for example, hundreds of thousands of collections in the database? If it's the viable solution to have a very large number of collections for a database for every tenant, what's the benefits of it for example versus having documents of each tenant in one collection? Thank you very much for your answers.

like image 706
Oleg Avatar asked Mar 25 '12 06:03

Oleg


People also ask

How many collections are allowed in MongoDB?

In general, try to limit your replica set to 10,000 collections.

How do you define maximum number of documents in collection?

Unless you create a capped collection, there is no limit to the number of documents in a collection. There is a 16MB limit to the size of a document (use gridfs in this situation) and limits in the storage engine for the size of the database and data.

What is the maximum size of index key limit and number of indexes per collection?

Explanation. The total size of an index entry, which can include structural overhead depending on the BSON type, must be less than 1024 bytes. A single collection can have no more than 64 indexes.

How many databases can MongoDB have?

If you have a free (M0) or shared tier cluster (M2, M5) there is currently a limit of 100 databases and 500 collections total. I would consider how you plan to model application users versus database users.


2 Answers

This answer is late however the other answers seem a bit...weak in terms of reliability and factual information so I will attempt to remedy that a little.

But for some reason mongodb set limit 24000 for the number of namespaces in the database,

That is merely the default setting. Yes, there is a default setting.

It does say on the limits page that 24000 is the limit ( http://docs.mongodb.org/manual/reference/limits/#Number%20of%20Namespaces ), as though there is no way to expand that but there is.

However there is a maximum limit on how big a namespace file can be ( http://docs.mongodb.org/manual/reference/limits/#Size%20of%20Namespace%20File ) which is 2GB. That gives you roughly 3 million namespaces to play with in most cases which is quite impressive and I am unsure if many people will hit that limit quickly.

You can modify the default value to go higher than 16MB by using the nssize parameter either within the configuration ( http://docs.mongodb.org/manual/reference/configuration-options/#nssize ) or at runtime by manipulating the command used to run MongoDB ( http://docs.mongodb.org/manual/reference/mongod/#cmdoption-mongod--nssize ).

There is no real reason for why MongoDB implements 16MB by default for its nssize as far as I know, I have never heard about the motto of "not bother the user with every single detail" so I don't buy that one.

I think, in my opinion, the main reason why MongoDB hides this is because even though, as the documentation states:

Distinct collections are very important for high-throughput batch processing.

Using multiple collections as a means to scale vertically rather than horizontally through a cluster, as MongoDB is designed to, is considered (quite often) bad practice for large scale websites; as such 12K collections is normally considered something that people will never, and should never, ascertain.

like image 68
Sammaye Avatar answered Sep 21 '22 23:09

Sammaye


No More Limits!

As other answers have stated - this is determined by the size of the namespace file. This was previously an issue, because it had a default limit of 16mb and a max of 2gb. However with the release of MongoDB 3.0 and the WiredTiger storage engine, it looks like this limit has been removed. WiredTiger seems to be better in almost every way, so I see little reason for anyone to use the old engine, except for legacy support reasons. From the site:

For the MMAPv1 storage engine, namespace files can be no larger than 2047 megabytes.

By default namespace files are 16 megabytes. You can configure the size using the nsSize option.

The WiredTiger storage engine is not subject to this limitation.

http://docs.mongodb.org/manual/reference/limits/

like image 39
user3413723 Avatar answered Sep 25 '22 23:09

user3413723