K8S: How to reach pod with SSH service (Gitea) over the same host ingress-nginx exposes as HTTP?

Question

Currently practicing with Kubernetes (managed, on DO), I ran into a issue I couldn't resolve for two days. I have nginx-ingress setup along with cert-manager, and a domain where git.domain.com points to the IP of the load balancer. I can reach my Gitea deployment via the web, everything seems to work.

What I want to achieve now is, that I can also use SSH like so

git clone [email protected]:org/repo.git

So I somehow need to expose the container port 22 via the service, then via the ingress. I tried a couple of things, but none of them seemed to work, probably because I'm a starter at K8S. Here is the working setup I use.

Service definition:

apiVersion: v1
kind: Service
metadata:
  name: gitea-service
spec:
  selector:
    app: gitea
  ports:
  - name: gitea-http
    port: 3000
    targetPort: gitea-http
  - name: gitea-ssh
    port: 22
    targetPort: gitea-ssh

Ingress definiton

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: echo-ingress
  annotations:  
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
  - hosts:
    - git.domain.com
    secretName: letsencrypt-prod
  rules:
  - host: git.domain.com
    http:
      paths:
      - backend:
          serviceName: gitea-service
          servicePort: gitea-http

And part of my deployment, just to make sure:

...
ports:
        - containerPort: 3000
          name: gitea-http
        - containerPort: 22
          name: gitea-ssh
...

Sorry if it's a dumb question, I think there is some basics that I confuse here. Thanks!

Answer 1

So I somehow need to expose the container port 22 via the service, then via the ingress

So yes and no: an Ingress is specifically for virtual-hosting using the host: header (or SNI) of the incoming request to know which backend to use. There is no such mechanism in SSH, or at the very least there's no Ingress controller that I'm aware of which supports protocols other than http for doing that.

However, the nginx Ingress controller supports TCP and UDP services so long as you can assign a dedicated port for them (which in your case, you can). You would create a ConfigMap entry saying which port on the ingress controller's Service to map to the port on gitea's Service, and then you'll need to expose port 22 on whatever is Internet-facing in Digital Ocean that routes traffic to the ingress controller's Service.

[Internet] -> :22[load balancer] --> :12345[controller Service] --> :22[gitea-service]

There are Digital Ocean annotations that you can use to switch certain ports over to TCP, but I didn't study that further than a quick search

I just used the nginx ingress controller as a concrete example, but the haproxy based ingress controllers will almost certainly do that, and other controllers may have similar options, because your question is a reasonable one