ingress-nginx - create one ingress per host? Or combine many hosts into one ingress and reload?

Question

I'm building a service where users can build web apps - these apps will be hosted under a virtual DNS name *.laska.io

For example, if Tom and Jerry both built an app, they'd have it hosted under:

tom.laska.io
jerry.laska.io

Now, suppose I have 1000 users. Should I create one big ingress that looks like this?

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - host: tom.laska.io
    http:
      paths:
      - backend:
          serviceName: nginx-service
          servicePort: 80
  - host: jerry.laska.io
    http:
      paths:
      - backend:
          serviceName: nginx-service
          servicePort: 80          
  ...and so forth             

I'm worried about downtime - if I have an app with websockets for example. Also the file will become huge with 1000 users. Will reloading the ingress go fast enough? Also, how should I reload it?

A second option in my mind is to simply create one ingress for every web app. My worry about that is, can ingress-nginx handle many ingresses? Or is this an anti-pattern?

Which one is better?

Answer 1

You can create one ingress resource for each web app. If you search the official public charts repo, you'll see that many of the charts define an ingress resource within them. It's normal for each app to define its own ingress resource.

It's worth being clear that an ingress resource is just a definition of a routing rule. (It doesn't add an extra ingress controller or any other extra runtime component.) So it makes a lot of sense for an app to define its own routing rule.

The example you've given has all the ingress routing in one resource definition. This approach is easy to grasp and makes a lot of sense when you've got several related applications as then you might want to see their routing configuration together. You can see this also in the fanout ingress example in the kubernetes docs.

I can't see any performance concerns with defining the rules separately in distinct resources. The ingress controller will listen for new rules and update its configuration only when a new rule is created so there shouldn't be a problem with reading the resources. And I'd expect the combined vs separated options to result in the same routing rules being set in the background in nginx.

The most common pattern in the official charts repo is that the chart for each app defines its ingress resource and also exposes it through the values.yaml so that users can choose to enable or customise it as they wish. You can then compose multiple charts together and configure the rules for each in the relevant section of the values.yaml. (Here's an example I've worked on that does this with wildcard dns.) Or you can deploy each app separately under its own helm release.

Answer 2

An ingress ressource is just a rule. It's better to split your ingress ressources in different files and just reapply the ressources that need change. I never noticed a downtime when applying a ressource.