Allow communication between two docker bridge networks using docker-compose

Question

I'm creating rather complex infrastructure of docker containers using docker compose. Those containers run in 4 different networks (similar to production environment that I'm mimicking). Docker Compose creates those four networks for me, and everything works as long, as containers don't try to communicate with other containers inside different networks. When they do, connection is dropped. I was able to find out why it is dropped, and it is because Docker adds iptables rules into DOCKER-ISOLATION chain. Example:

-A DOCKER-ISOLATION -i br-be010eaddd0e -o br-f788f16ed0dd -j DROP
-A DOCKER-ISOLATION -i br-f788f16ed0dd -o br-be010eaddd0e -j DROP

I wrote a little script that removes rules I want to be removed (and allow communication between chosen bridges) and everything works like a charm, but somehow they are recreated by Docker at some point, even without recreating those networks, so it requires me to run that script again, which is very annoying. Is there any way to specifically tell Docker to allow communication between two bridges? Or maybe there's some trick to run specific shell script after starting containers with Docker-Compose?

Answer 1

If anyone's interested, I've managed to handle this with iptables. Solution is to explicitly allow communication between bridge subnets (assuming that they have fixed ip addresses). The way to do this is to issue following commands (assuming that bridge subnets are 172.24.131.0/24 and 172.24.132.0/24):

iptables -I FORWARD -s 172.24.131.0/24 -d 172.24.132.0/24 -j ACCEPT
iptables -I FORWARD -d 172.24.131.0/24 -s 172.24.132.0/24 -j ACCEPT

That way we add new rules in FORWARD chain just before DOCKER-ISOLATION, it forces iptables to ignore whole DOCKER-ISOLATION chain for any communication between these subnets.

Answer 2

Isolation is the core reason to use multiple bridge networks. In Docker world you should put the containers you would like to communicate with each other in the same network.

I assume you would like to replicate the production network with 4 separate subnets/layer 2 domains that use a router to communicate with each other for testing and other purposes? Try using the experimental ipvlan driver to form the ipvlan L3 network with 4 different subnets and ipvlan driver serving as a router between them. Here is how you set it up (scroll down to ipvlan L3.