I spinned a docker-openvpn
container in my (local) Kubernetes cluster to access my Services securely and debug dependent services locally.
I can connect to the cluster via the openVPN server. However I can't resolve my Services via DNS.
I managed to get to the point where after setting routes on the VPN server:
subnet 10.2.0.0/16
)subnet 10.3.0.0/16
like the DNS which is at 10.3.0.10
)curl
to a Services by IP and get the data I need.but when i nslookup kubernetes
or any Service, I get:
nslookup kubernetes
;; Got recursion not available from 10.3.0.10, trying next server
;; Got SERVFAIL reply from 10.3.0.10, trying next server
I am still missing something for the data to return from the DNS server, but can't figure what I need to do.
How do I debug this SERVFAIL
issue in Kubernetes DNS?
EDIT:
Things I have noticed and am looking to understand:
nslookup
works to resolve Service name in any pod except the openvpn Podnslookup
works in those other Pods, ping
does not.traceroute
in those other Pods leads to the flannel layer 10.0.2.2
and then stops there.from this I guess ICMP must be blocked at the flannel layer, and that doesn't help me figure where DNS is blocked.
EDIT2:
I finally figured how to get nslookup to work: I had to push the DNS search domain to the client with
push "dhcp-option DOMAIN-SEARCH cluster.local"
push "dhcp-option DOMAIN-SEARCH svc.cluster.local"
push "dhcp-option DOMAIN-SEARCH default.svc.cluster.local"
add with the -p
option in the docker-openvpn
image
so i end up with
docker run -v /etc/openvpn:/etc/openvpn --rm kylemanna/openvpn ovpn_genconfig \
-u udp://192.168.10.152:1194 \
-n 10.3.0.10 \
-n 192.168.10.1 \
-n 8.8.8.8 \
-n 75.75.75.75 \
-n 75.75.75.76 \
-s 10.8.0.0/24 \
-d \
-p "route 10.2.0.0 255.255.0.0" \
-p "route 10.3.0.0 255.255.0.0" \
-p "dhcp-option DOMAIN cluster.local" \
-p "dhcp-option DOMAIN-SEARCH svc.cluster.local" \
-p "dhcp-option DOMAIN-SEARCH default.svc.cluster.local"
Now, nslookup
works but curl
still does not
finally my config looks like this:
docker run -v /etc/openvpn:/etc/openvpn --rm kylemanna/openvpn ovpn_genconfig \
-u udp://192.168.10.152:1194 \
-n 10.3.0.10 \
-n 192.168.10.1 \
-n 8.8.8.8 \
-n 75.75.75.75 \
-n 75.75.75.76 \
-s 10.8.0.0/24 \
-N \
-p "route 10.2.0.0 255.255.0.0" \
-p "route 10.3.0.0 255.255.0.0" \
-p "dhcp-option DOMAIN-SEARCH cluster.local" \
-p "dhcp-option DOMAIN-SEARCH svc.cluster.local" \
-p "dhcp-option DOMAIN-SEARCH default.svc.cluster.local"
-u
for the VPN server address and port
-n
for all the DNS servers to use
-s
to define the VPN subnet (as it defaults to 10.2.0.0 which is used by Kubernetes already)
-d
to disable NAT
-p
to push options to the client
-N
to enable NAT: it seems critical for this setup on Kubernetes
the last part, pushing the search domains to the client, was the key to getting nslookup
etc.. to work.
note that curl didn't work at first, but seems to start working after a few seconds. So it does work but it takes a bit of time for curl to be able to resolve.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With