Max number of socket on Linux

Question

It seems that the server is limited at ~32720 sockets... I have tried every known variable change to raise up this limit. But the server stay limited at 32720 opened socket, even if there is still 4Go of free memory and 80% of idle cpu...

Here's the configuration

~# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 63931
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 798621
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 2048
cpu time               (seconds, -t) unlimited
max user processes              (-u) 63931
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

net.netfilter.nf_conntrack_max = 999999
net.ipv4.netfilter.ip_conntrack_max = 999999
net.nf_conntrack_max = 999999

Any thoughts ?

Answer 1

If you're dealing with openssl and threads, go check your /proc/sys/vm/max_map_count and try to raise it.

Answer 2

In IPV4, the TCP layer has 16 bits for the destination port, and 16 bits for the source port.

see http://en.wikipedia.org/wiki/Transmission_Control_Protocol

Seeing that your limit is 32K I would expect that you are actually seeing the limit of outbound TCP connections you can make. You should be able to get a max of 65K sockets (this would be the protocol limit). This is the limit for total number of named connections. Fortunately, binding a port for incoming connections only uses 1. But if you are trying to test the number of connections from the same machine, you can only have 65K total outgoing connections (for TCP). To test the amount of incoming connections, you will need multiple computers.

Note: you can call socket(AF_INET,...) up to the number of file descriptors available, but you cannot bind them without increasing the number of ports available. To increase the range, do this:

echo "1024 65535" > /proc/sys/net/ipv4/ip_local_port_range (cat it to see what you currently have--the default is 32768 to 61000)

Perhaps it is time for a new TCP like protocol that will allow 32 bits for the source and dest ports? But how many applications really need more than 65 thousand outbound connections?

The following will allow 100,000 incoming connections on linux mint 16 (64 bit) (you must run it as root to set the limits)

#include <stdio.h>
#include <sys/time.h>
#include <sys/resource.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/ip.h>

void ShowLimit()
{
   rlimit lim;
   int err=getrlimit(RLIMIT_NOFILE,&lim);
   printf("%1d limit: %1ld,%1ld\n",err,lim.rlim_cur,lim.rlim_max);
}

main()
{
   ShowLimit();

   rlimit lim;
   lim.rlim_cur=100000;
   lim.rlim_max=100000;
   int err=setrlimit(RLIMIT_NOFILE,&lim);
   printf("set returned %1d\n",err);

   ShowLimit();

   int sock=socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
   sockaddr_in maddr;
   maddr.sin_family=AF_INET;
   maddr.sin_port=htons(80);
   maddr.sin_addr.s_addr=INADDR_ANY;

   err=bind(sock,(sockaddr *) &maddr, sizeof(maddr));

   err=listen(sock,1024);

   int sockets=0;
   while(true)
   {
      sockaddr_in raddr;
      socklen_t rlen=sizeof(raddr);
      err=accept(sock,(sockaddr *) &raddr,&rlen);
      if(err>=0)
      {
        ++sockets;
        printf("%1d sockets accepted\n",sockets);
      }
   }
}