Docker Swarm mode routing mesh not working with wireguard VPN

Question

I'm trying to setup a 3 node Docker swarm cluster on Hetzner cloud, using wireguard VPN (setup on interface wg0) to build the local network between nodes. Networking works fine across nodes using VPN IP (ports 7946/tcp , 7946/udp and 4789/udp are open as reported here). I start docker swarm cluster with the following commands:

docker swarm init --advertise-addr wg0 --listen-addr wg0
docker swarm join --token SWMTKN-1-xxx --advertise-addr wg0 --listen-addr wg0 10.0.0.1:2377

If I try to run a service on this swarm, it seems to run correctly, every container can reach the other on different node and inspecting them, they join the ingress network and an overlay network created by me, as expected. The problem arises when I try to access from outside the service exposed port; it only works if I target the node where the container is running, so it seems that the routing mesh is not correctly working. I've not found any error in docker logs or syslog.

Note: I'm using docker 18.06.1-ce

Answer 1

I had this issue and made the following changes:

1. I moved my wireguard addresses from 10.0.* to 192.168.* (i have a feeling that swarm is allocating on top of these).
2. docker swarm init --advertise-addr 192.168.2.123 with the wireguard ip4 address of the master node.

That managed to fix it, and it still works after rebooting the master and worker nodes!