Why does Docker run so many processes to map ports though to my application?

Question

I have an application that runs in a container that requires a range of ports to be mapped to it.

docker run -p 2000-3000:2000-3000 myapp

When I run this docker command my development vm grinds to a halt.

Then looking at the processes, there is a docker-proxy running for every port

$ ps -ef 
...
root     19796  7835  0 03:31 ?        00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 4000 -container-ip 172.17.0.4 -container-port 3000
root     19804  7835  0 03:31 ?        00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 3999 -container-ip 172.17.0.4 -container-port 2999
root     19812  7835  0 03:31 ?        00:00:00 docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 3998 -container-ip 172.17.0.4 -container-port 2998
...

$ ps -ef | grep -c docker-proxy
1003

They are all children of the docker daemon

root@default-docker:~# pstree -p
init(1)-+-VBoxService(1251)
        |-acpid(1277)
        |-crond(1235)
        |-docker(7835)-+-docker-containe(7841)-+-docker-containe(8031)---gitlab-ci-multi(8048)
        |              |                       |-docker-containe(9678)---mysqld(9693)
        |              |                       `-docker-containe(20577)---registry(20591)
        |              |-exe(19796)
        |              |-exe(19804)
        |              |-exe(19812)

Each process uses a chunk of private memory (Pss in /proc/$pid/smaps)

$ for pid in $(pgrep exe); do printf "pid:%5s mem:%5s\n" $pid $(awk '/^Pss:/{t=t+$2}END{print t}' /proc/$pid/smaps); done
...
pid:28534 mem: 4011
pid:28543 mem: 3817
pid:28552 mem: 4001

There are also DNAT rules in place for each port, which is how I would have expected this to be done on a Linux host with private networks.

root@default-docker:~# iptables -t nat -vnL DOCKER
Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
...
    0     0 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:4000 to:172.17.0.4:3000
    0     0 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3999 to:172.17.0.4:2999
    0     0 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:3998 to:172.17.0.4:2998
...

Why is Docker launching a process per port?
Why does each process need 4-6MB of memory?
Why is Docker using a user space process at all?

Answer 1

Why is Docker using a user space process at all?

Nigel Brown has written a detailed article on The docker-proxy which explains the how and why.

The docker-proxy, then, is a 'catch all' method for allowing container port forwarding to the Docker host. However, it's generally considered that the docker-proxy is an inelegant solution to the problems highlighted above, and when a large range of container ports are exposed, it consumes considerable memory. An attempt was previously made to remove the dependency for the docker-proxy, but this fell foul of the limitations of the aged kernel in RHEL 6.x and CentOS 6.x, which the Docker project feels duty bound to support. Hence, the docker-proxy remains a major constituent part of the Docker experience in all Docker versions up to the current version 1.5. As I write, version 1.6 is due for imminent release, and there have been moves to remove the automatic requirement for the docker-proxy, which I'll cover in another article.

Docker now includes a daemon run time option to disable the userland proxy with --userland-proxy=false. This was introduced in v1.7.

There seems to be a few edge case bugs that exist when disabling the userland proxy. There are also IPV6 issues

There is an open GitHub issue for disabling the userland proxy by default (RHEL6 is no longer supported by Docker).

Why is Docker launching a process per port?

There doesn't appear to be a reason for this other than it was implemented this way. A single process should be capable of handling all of the port mappings for a container

Why does each process need 4-6MB of memory?

The proxy implementation and package looks clean and uses in built Go functionality so this might just be Go's initial garbage collection limits that allow it to grow to ~ 5MB.

EDIT: Memory usage has been improved in Docker 1.12. There is still a process per port but each process now only uses ~ 750k of private memory space.