How to resolve pod hostnames from other pods?

Question

• I have 2 pods running on 2 nodes, each pod runs in different node.
• These nodes are on the same subnet and can TCP/UDP/ICMP themselves.

These pods got some hostnames, ie:

• drill-staging-75cddd789-kbzsq
• drill-staging-75cddd789-amsrj

From pod drill-staging-75cddd789-kbzsq I cannot resolve host name for drill-staging-75cddd789-amsrj and vice versa. Resolving self pod's name works.

I tried setting various dnsPolicies:

• ClusterFirst: no luck
• Default: no luck
• ClusterFirstWithHostNet: no luck and it event couldn't resolve hostname of it's own node
• None: not tried (I don't think it's a good way)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "app.name" . }}
  namespace: {{ .Values.global.namespace }}
spec:
  replicas: 2
  selector:
    matchLabels:
      app: {{ include "app.name" . }}
  template:
    metadata:
      labels:
        app: {{ include "app.name" . }}
    spec:
      containers:
      - name: {{ include "app.name" . }}
        image: ...
        resources:
          ...
        ports:
          ...
        imagePullPolicy: Always
      restartPolicy: Always

Answer 1

Normally, only Services get DNS names, not Pods. So, by default, you can't refer to another Pod directly by a domain name, only by its IP address.

Pods get DNS names only under certain condidtions that include a headless Service, as explained in the documentation. In particular, the conditions are:

• The Pods have a hostname field
• The Pods have a subdomain field
• There is a headless Service (in the same namespace) that selects the Pods
• The name of the headless Service equals the subdomain field of the Pods

In this case, each Pod gets a fully-qualified domain name of the following form:

my-hostname.my-subdomain.default.svc.cluster.local

Where my-hostname is the hostname field of the Pod and my-subdomain is the subdomain field of the Pod.

Note: the DNS name is created for the "hostname" of the Pod and not the "name" of the Pod.

You can test this with the following setup:

apiVersion: v1
kind: Service
metadata:
  name: my-subdomain
spec:
  selector:
    name: my-test
  clusterIP: None
---
apiVersion: v1
kind: Pod
metadata:
  name: my-pod-1
  labels:
    name: my-test
spec:
  hostname: my-hostname-1
  subdomain: my-subdomain
  containers:
  - image: weibeld/ubuntu-networking
    command: [sleep, "3600"]
    name: ubuntu-networking
---
apiVersion: v1
kind: Pod
metadata:
  name: my-pod-2
  labels:
    name: my-test
spec:
  hostname: my-hostname-2
  subdomain: my-subdomain
  containers:
  - image: weibeld/ubuntu-networking
    command: [sleep, "3600"]
    name: ubuntu-networking

After applying this, you can exec into one of the Pods:

kubectl exec -ti my-pod-1 bash

And you should be able to resolve the fully-qualifed domain names of the two Pods:

host my-hostname-1.my-subdomain.default.svc.cluster.local
host my-hostname-2.my-subdomain.default.svc.cluster.local

Since you're making the requests from the same namespace as the target Pods, you can abbreviate the domain name to:

host my-hostname-1.my-subdomain
host my-hostname-2.my-subdomain