EDIT: I now have a solution, but I'd really apprecite a concise description of what the different limits are, i.e. those set by FD_SIZE, launchtl limit files, sysctl -w kern.maxfilesperproc, ulimit etc.)
Can someone help me understand the limits on open filehandles on OSX. ulimit gives me one answer:
$ ulimit -a
...
open files (-n) 256
I can't use ulimit
to change this, but people suggest using launchctl
(e.g. http://usrinapto.wordpress.com/2010/03/06/mac-os-x-10-6-max-open-files-too-many-open-files/)
Using this doesn't change the limit reported by ulimit
, though.
However, my application seems to able to open 10k files before crashing, as reported by lsof
, e.g.:
$ lsof -p 87599 | wc
10279 92505 1418903
(it crashes somewhere between 10279 and 10305 open files, reliably)
So there are clearly different limits coming in to play. I've also seen talk (on the above link) of FD_SETSIZE
.
Can someone explain to me what the different limits are, and how they are set?
In case it's relevant, I'm working on wrapping a C/C++ library for use in Java, using SWIG.
EDIT: I've also tried:
sudo sysctl -w kern.maxfiles=20000
with no success. Also
#define FD_SETSIZE 20000
has no effect.
EDIT: Also tried
launchctl limit maxfiles 20000 20000
with no effect.
EDIT: Solution:
sysctl -w kern.maxfilesperproc=20000
(via http://krypted.com/mac-os-x/maximum-files-in-mac-os-x/)
EDIT: I've written a small program to test this (based on How to increase the limit of "maximum open files" in C on Mac OS X), and found that the max number of open files I can ask for is 10240:
#include <sys/resource.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
struct rlimit limit;
void setLimit( int l );
void getLimit();
int main( int argc, char* argv[] )
{
getLimit();
setLimit(10240);
getLimit();
return 1;
}
void setLimit( int lim )
{
limit.rlim_cur = lim;
limit.rlim_max = lim;
printf( "Setting limit to %d, %d\n", limit.rlim_cur, limit.rlim_max );
if (setrlimit(RLIMIT_NOFILE, &limit) != 0) {
printf("setrlimit() failed with errno=%d\n", errno);
exit(1);
}
}
void getLimit()
{
/* Get max number of files. */
if (getrlimit(RLIMIT_NOFILE, &limit) != 0)
{
printf("getrlimit() failed with errno=%d\n", errno);
exit(1);
}
printf("The soft limit is %llu\n", limit.rlim_cur);
printf("The hard limit is %llu\n", limit.rlim_max);
}
A simple fix for the "too many files open" limitation of Mac OS is to use the "ulimit - n" command. Curiously, the value of n appears to be critical to whether or not this command is accepted by MacOS. I've found that ulimit -n 10240 (the default is 256) works but n values higher do not.
found on http://krypted.com/mac-os-x/maximum-files-in-mac-os-x/:
sysctl -w kern.maxfilesperproc=20000
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With