Menu
Kubernetes/Ubuntu newbie here!
I'm setting up a k8s cluster using a single Raspberry Pi (hoping to have more in the future). I'm using microk8s v1.18.8 and Ubuntu Server 20.04.1 LTS (GNU/Linux 5.4.0-1018-raspi aarch64).
I'm trying to access one of my k8s services on port 80, but I haven't been able to set it up correctly. I've also set a static IP address for accessing the service, and I'm routing traffic from the router to the service's IP address.
I would like to know what I'm doing wrong, or if there's a better approach for what I'm trying to do!
The steps I'm following:
microk8s enable dns metallb. I've given MetalLB IP addresses not being handled by the DHPC server (192.168.0.90-192.168.0.99).ingress-nginx by running kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/baremetal/deploy.yaml. This creates a NodePort service for the ingress-nginx-controller, which doesn't work with MetalLB. As mentioned here, I edit the spec.type of the service from NodePort to LoadBalancer by running kubectl edit service ingress-nginx-controller -n ingress-nginx. MetalLB then assigns IP 192.168.0.90 to the service.apiVersion: v1
kind: Service
metadata:
name: wow-ah-api-service
namespace: develop
spec:
selector:
app: wow-ah-api
ports:
- protocol: TCP
port: 80
targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
# Unique key of the Deployment instance
name: wow-ah-api
namespace: develop
spec:
# 3 Pods should exist at all times.
replicas: 3
selector:
matchLabels:
app: wow-ah-api
template:
metadata:
namespace: develop
labels:
# Apply this label to pods and default
# the Deployment label selector to this value
app: wow-ah-api
spec:
imagePullSecrets:
- name: some-secret
containers:
- name: wow-ah-api
# Run this image
image: some-image
imagePullPolicy: Always
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: wow-ah-api-ingress
namespace: develop
spec:
backend:
serviceName: wow-ah-api-service
servicePort: 3000
These are some outputs I'm seeing:
microk8s kubectl get all --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
develop pod/wow-ah-api-6c4bff88f9-2x48v 1/1 Running 4 4h21m
develop pod/wow-ah-api-6c4bff88f9-ccw9z 1/1 Running 4 4h21m
develop pod/wow-ah-api-6c4bff88f9-rd6lp 1/1 Running 4 4h21m
ingress-nginx pod/ingress-nginx-admission-create-mnn8g 0/1 Completed 0 4h27m
ingress-nginx pod/ingress-nginx-admission-patch-x5r6d 0/1 Completed 1 4h27m
ingress-nginx pod/ingress-nginx-controller-7896b4fbd4-nglsd 1/1 Running 4 4h27m
kube-system pod/coredns-588fd544bf-576x5 1/1 Running 4 4h26m
metallb-system pod/controller-5f98465b6b-hcj9g 1/1 Running 4 4h23m
metallb-system pod/speaker-qc9pc 1/1 Running 4 4h23m
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 21h
develop service/wow-ah-api-service ClusterIP 10.152.183.88 <none> 80/TCP 4h21m
ingress-nginx service/ingress-nginx-controller LoadBalancer 10.152.183.216 192.168.0.90 80:32151/TCP,443:30892/TCP 4h27m
ingress-nginx service/ingress-nginx-controller-admission ClusterIP 10.152.183.41 <none> 443/TCP 4h27m
kube-system service/kube-dns ClusterIP 10.152.183.10 <none> 53/UDP,53/TCP,9153/TCP 4h26m
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
metallb-system daemonset.apps/speaker 1 1 1 1 1 beta.kubernetes.io/os=linux 4h23m
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
develop deployment.apps/wow-ah-api 3/3 3 3 4h21m
ingress-nginx deployment.apps/ingress-nginx-controller 1/1 1 1 4h27m
kube-system deployment.apps/coredns 1/1 1 1 4h26m
metallb-system deployment.apps/controller 1/1 1 1 4h23m
NAMESPACE NAME DESIRED CURRENT READY AGE
develop replicaset.apps/wow-ah-api-6c4bff88f9 3 3 3 4h21m
ingress-nginx replicaset.apps/ingress-nginx-controller-7896b4fbd4 1 1 1 4h27m
kube-system replicaset.apps/coredns-588fd544bf 1 1 1 4h26m
metallb-system replicaset.apps/controller-5f98465b6b 1 1 1 4h23m
NAMESPACE NAME COMPLETIONS DURATION AGE
ingress-nginx job.batch/ingress-nginx-admission-create 1/1 27s 4h27m
ingress-nginx job.batch/ingress-nginx-admission-patch 1/1 29s 4h27m
microk8s kubectl get ingress --all-namespaces
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
develop wow-ah-api-ingress <none> * 192.168.0.236 80 4h23m
I have been thinking it could be related to my iptables configuration, but I'm not sure how to configure them to work with microk8s.
sudo iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
KUBE-EXTERNAL-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes externally-visible service portals */
KUBE-FIREWALL all -- anywhere anywhere
Chain FORWARD (policy ACCEPT)
target prot opt source destination
KUBE-FORWARD all -- anywhere anywhere /* kubernetes forwarding rules */
KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
ACCEPT all -- 10.1.0.0/16 anywhere /* generated for MicroK8s pods */
ACCEPT all -- anywhere 10.1.0.0/16 /* generated for MicroK8s pods */
ACCEPT all -- 10.1.0.0/16 anywhere
ACCEPT all -- anywhere 10.1.0.0/16
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
KUBE-FIREWALL all -- anywhere anywhere
Chain KUBE-EXTERNAL-SERVICES (1 references)
target prot opt source destination
Chain KUBE-FIREWALL (2 references)
target prot opt source destination
DROP all -- anywhere anywhere /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
DROP all -- !localhost/8 localhost/8 /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT
Chain KUBE-FORWARD (1 references)
target prot opt source destination
DROP all -- anywhere anywhere ctstate INVALID
ACCEPT all -- anywhere anywhere /* kubernetes forwarding rules */ mark match 0x4000/0x4000
ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-PROXY-CANARY (0 references)
target prot opt source destination
Chain KUBE-SERVICES (3 references)
target prot opt source destination
UPDATE #1
metallb ConfigMap (microk8s kubectl edit ConfigMap/config -n metallb-system)
apiVersion: v1
data:
config: |
address-pools:
- name: default
protocol: layer2
addresses:
- 192.168.0.90-192.168.0.99
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"config":"address-pools:\n- name: default\n protocol: layer2\n addresses:\n - 192.168.0.90-192.168.0.99\n"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"config","namespace":"metallb-system"}}
creationTimestamp: "2020-09-19T21:18:45Z"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:config: {}
f:metadata:
f:annotations:
.: {}
f:kubectl.kubernetes.io/last-applied-configuration: {}
manager: kubectl
operation: Update
time: "2020-09-19T21:18:45Z"
name: config
namespace: metallb-system
resourceVersion: "133422"
selfLink: /api/v1/namespaces/metallb-system/configmaps/config
uid: 774f6a73-b1e1-4e26-ba73-ef71bc2e1060
I'd appreciate any help you could give me!
914
When you use an ingress controller and ingress rules, a single IP address can be used to route traffic to multiple services in a Kubernetes cluster. This article shows you how to deploy the NGINX ingress controller in an Azure Kubernetes Service (AKS) cluster.
Coming to your query Ingress-nginx is not a load balancer but on a broader lever can help you with load balancing.
Clean Microk8s. Only one public IP (or local machine IP. In your use case, I'll use 192.168.0.90).
How do you test? For example
curl -H "Host: blue.nginx.example.com" http://PUBLIC_IP
from outside the machine.
Run the test. It must fail.
Enable microk8s dns and ingress
microk8s.enable dns ingress
Run the test. Fails?
If it's the same error then: You need metallb
With Internet public IP
microk8s.enable metallb:$(curl ipinfo.io/ip)-$(curl ipinfo.io/ip)
With LAN IP 192.168.0.90
microk8s.enable metallb:192.168.0.90-192.168.0.90
Run the test again
If Test NOT return 503 or 404 then: You can't do next steps. Perhaps you have a network problem or firewall filter.
Our test arrived to the Microk8s Ingress controller. He doesn't know what to do and return a 404 error (sometimes 503).
That's ok. Go next!
I'll use an example from https://youtu.be/A_PjjCM1eLA?t=984 16:24
[ Kube 32 ] Set up Traefik Ingress on kubernetes Bare Metal Cluster
set kubectl alias
alias kubectl=microk8s.kubectl
kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/nginx-deploy-main.yaml
kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/nginx-deploy-blue.yaml
kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/nginx-deploy-green.yaml
kubectl expose deploy nginx-deploy-main --port 80
kubectl expose deploy nginx-deploy-blue --port 80
kubectl expose deploy nginx-deploy-green --port 80
Run Test. It doesn't work... yet.
Ingress rule example: how to delivery by host nameConfigure the hosts nginx.example.com, blue.nginx.example.com, and green.nginx.example.com and distribute requests to exposed deployments:
kubectl create -f https://raw.githubusercontent.com/justmeandopensource/kubernetes/master/yamls/ingress-demo/ingress-resource-2.yaml
Run this tests:
curl -H "Host: blue.nginx.example.com" http://PUBLIC_IP
Now you'll have a response like
<h1>I am <font color=blue>BLUE</font></h1>
You can play with
curl -H "Host: nginx.example.com" http://PUBLIC_IP
curl -H "Host: blue.nginx.example.com" http://PUBLIC_IP
curl -H "Host: green.nginx.example.com" http://PUBLIC_IP
Just started with MicroK8s - it appears to have great promise. After combing through info sites and docs; was able to implement bare metal demonstration with Traefik Ingress Controller (with Custom Resource Definitions and Ingress Routes); Linkerd service mesh; and metallb load balancer. This was done on a VirtualBox Guest VM running Ubuntu 20.04; also included with this github link is "way" to expose Traefik Ingress Controller external IP provided by metallb external to Guest VM. See https://github.com/msb1/microk8s-traefik-linkerd-whoami .
Prefer this implementation to what is shown in Youtube link as it includes working service mesh and uses custom resource definitions for Ingress (which is unique to Traefik and one of reasons to proceed with Traefik as opposed to other Ingress Controllers).
Hope this help others - should be able to build awesome deployments with MicroK8s following this demo (which is current focus).
28
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
