Mongorestore - too many open files - mac os x

Question

I tried to import dump of a big database to my local instance of mongodb.

Unfortunatelly I found a problem that for one of imported collection, mongo threw exception, that too many files are open.

I went throught the mighty knowledge of the internet using google and I found some solutions with ulimit and launchctl, but they didn't work.

Finally I resolved the problem in the following way:

1. I created /etc/launchd.conf file and added there these lines:

    limit maxproc 512 1024
    limit maxfiles 16384 32768

2. Next I execute follwing lines in the terminal:

    sudo sysctl -w kern.maxfilesperproc=16384
    sudo sysctl -w kern.maxfiles=32768

3. Finally I restarted the OS.

The issue doesn't occur anymore, but I have question. If there is some solution to limit the number of opened files from the mongorestore level ? I don't think that increasing a global value for max opened files is a good way.

Answer 1

Indeed --numParallelCollections=1 fixed this for me on OS-X, without modifying system settings. I was able to do a full DB restore which previously didn't complete. However it seems to max out the connection pool as I still get 2017-06-01T16:55:19.386+0800 E NETWORK [initandlisten] Out of file descriptors. Waiting one second before trying to accept more connections. when trying to continue. required to restart mongod

Answer 2

It's much safer to change the limit only on the current shell session. Adding it to your bash profile is also an option if you need to have it on every bash console session.

ulimit -S -n 2048
your_whatever_greedy_command...

You may modify 2048 with another value. If your greedy command is starting mongo, it will be:

ulimit -S -n 2048 && mongod