OSError: [Errno 24] Too many open files - OS Mojave

Question

So, I'm writing a script to take a certain dataset, sample it 100 times using different random seeds, completing all these datasets and then getting the mean error. However, whenever I try to run the script I end up with the error OSError: [Errno 24] Too many open files

I don't understand what I can do to fix this (if I should do something in the script or in to the system and what). I'm using Python 3 with macOS Mojave. Anybody has a clue?

Answer 1

The number of open files you're allowed can be increased by using ulimit e.g. in bash you could do: ulimit -n This will probably print out 256 meaning that at one time a maximum of 256 file descriptors are allowed to be open.

Increase the limit: ulimit -n 30000 # 30,000 open files allowed

This sort of thing is generally done on systems running something like server programs that need a file descriptor (a socket descriptor) for each concurrent connection being handled.

However, if you post the code there might be another way to fix this. It doesn't sound like you want/need multiple file descriptors open at the same time.