How to connect multiple Dockers to different bridges in a single host machine?

Question

Is it possible to connect multiple Dockers (in my case, multiple instances of the same docker) each to different bridge in a single host machine?

3rd party solutions like pipework recommends first looking for "native" ways.

Something like that:

When I start 2 instance, the 1st use by default docker0 bridge, while the 2nd is instructed to use br1 (different IP range) :

sudo docker run -t -i me/tester:latest /bin/bash
sudo docker --bridge=br1 run -t -i me/tester:latest /bin/bash

results in both having the same IP range from Doker0:

root@2a259a88d9c8:/# ip a
...
73: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 86:d7:cc:c8:b7:e8 brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.32/16 scope global eth0

--

root@0b849a5398af:/# ip a
...
79: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 26:84:ad:6c:70:6b brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.34/16 scope global eth0
       valid_lft forever preferred_lft forever

Answer 1

This is how it is done using the new native docker networking:

Create docker networks (linux bridges) with a predefined subnets

docker network create --subnet=192.168.10.0/24 net1
docker network create --subnet=192.168.20.0/24 net2
docker network create --subnet=192.168.30.0/24 net3

The created networks correspond to linux bridges

brctl show

Create tap interfaces

sudo ip tuntap add dev tap1 mode tap
sudo ip tuntap add dev tap2 mode tap
sudo ip tuntap add dev tap3 mode tap

and join them to the bridges

sudo brctl addif br-a24f2eb2e054 tap1
sudo brctl addif br-d28c0759c37a tap2
sudo brctl addif br-d9512f62e471 tap3

starting your containers

sudo docker run -itd --name=c1  phusion/baseimage
sudo docker run -itd --name=c2  phusion/baseimage
sudo docker run -itd --name=c3  phusion/baseimage

Connecting containers to network

docker network connect net1 c1
docker network connect net2 c2
docker network connect net3 c3

Verify that each container is connected to its network

docker network inspect net1

docker network inspect net2

docker network inspect net3

Connected containers get their ip from their corresponding network subnets

docker exec c1 ip a s eth1

docker exec c2 ip a s eth1

docker exec c3 ip a s eth1

Disconnecting containers from networks

docker network disconnect net1 c1
docker network disconnect net2 c2
docker network disconnect net3 c3

Remove the networks

docker network rm net1
docker network rm net2
docker network rm net3

Answer 2

You can create custom bridges and then run each container with the option -b <BRIDGE> or --bridge=<BRIDGE>, but I have not try this and I wouldn't be absolutely sure that this approach works without any issue. If you read this Docker issue, I think the suggest it is not possible this way.

But as commented in that issue, you can disable any docker network management (running the container with the option --net=none, and then organize the network as you wish using pipework.

Finally you can take a look of zettio/weave. Using it you can run easily each container in the network you wish, and also it gives you a lot of possibilities if you want to expand your docker environment from one to various machines (as docker swarm does).