Mosquitto increasing maximum connection limit

Question

I am trying to increase the maximum open file connection of Mosquitto broker. But I read that increasing concurrent connections are not controlled by Mosquitto only. As per our study we decided for 100k concurrent connection, we are targeting 1.6 GB RAM. But for testing I have to increase from default 1024 connections to 20000 Testing environment configurations: t2. micro AWS server with 64 MB 14.04 ubuntu operating system. Changing connection limit in the mosquitto configuration is not reflecting. What can be the reason? Do we need to change any configuration related to AWS Server?

My configurations:

Our system wide open connections is configured on /etc/sysctl.conf:

fs.file-max =99905

Running the command sysctl -p or cat /proc/sys/fs/file-max is reflecting the changes

In /etc/security/limits.conf:

ubuntu          hard     nofile         45000
ubuntu          soft     nofile         35000

Mosquitto is installed under the user 'Ubuntu' .

We also added below line of code on /etc/pam.d/common-session

session required pam_limits.so

Running ulimit -a is giving the below result:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7859
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 35000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7859
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

My init configuration file for mosquitto in /etc/init/mosquitto.conf

description "Mosquitto MQTTv3.1 broker"
author "Roger Light <[email protected]"

start on net-device-up

respawn

exec /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
#limit nofile 4096 4096

limit nofile 24000  24000

Below is the configuration in /etc/mosquitto/mosquitto.conf:

# change user to root
user ubuntu         

set_ulimit () {
    ulimit -f unlimited
    ulimit -t unlimited
    ulimit -v unlimited
    ulimit -n 24000
    ulimit -m unlimited
    ulimit -u 24000
}

start)
    ...
    # Update ulimit config in start command
    set_ulimit
    ...
    ;;
  stop)

But running cat /proc/4957/limits is still showing default value 1024 open files:

Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             7859                 7859                 processes
Max open files            1024                 4096                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       7859                 7859                 signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

4957 -is the process id of Mosquitto

Answer 1

The number of open files is limited by the user limits, see ulimit man page. I set the ulimit -n to 20000, and run mosquitto broker and it shows

% ps ax | grep mosquitto 9497 pts/44 S+ 0:00 ./mosquitto -c mosquitto.conf 9505 pts/10 S+ 0:00 grep --color=auto mosquitto % cat /proc/9497/limits Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 63084 63084 processes Max open files 20000 20000 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 63084 63084 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us

Regardless, since mosquitto is single-threaded, we have not found it useable for anything more than about 1000 publisher clients with a reasonable payload rate at 1 / 10 seconds.

Answer 2

changing limits in /etc/sysctl.conf or /etc/security/limits.conf seems to have no effect for process launched as service: The limit has to be set in the file starting up the daemon.

At the beginning of /etc/init.d/mosquitto: ulimit -n 20000 #or more if need more....

in /etc/mosquitto/mosquitto.conf: max_connections -1 #or the max number of connection you want