Upgrading from Helm stable/cert-manager to jetstack/cert-manager

Question

We have a production AKS cluster that has a stable/cert-manager helm chart installed to allow using Let's Encrypt certificates. The current version installed is cert-manager-v0.6.0 in the kube-system namespace.

Let's Encrypt is to stop support for traffic from cert-manager pre 8.0 version from 1st of November 2019.

I would like to upgrade but the latest available stable chart version is v0.6.7. Seems like the way to go is to switch to jetstack/cert-manager.

How do I best approach this? Shall I uninstall the current stable/cert-manager chart and install from scratch with the jetstack/cert-manager? Any resource on how to tackle this without downtime in production would be much appreciated. Please let me know if I can provide any more details.

Answer 1

For anyone asking the same question, I have tried to perform clean install on my test cluster and this seemed to work fairly smoothly. I have found what the name of my the helm release was by running helm list

then I have performed the following steps:

1.Backup:

kubectl get -o yaml \
   --all-namespaces \
   issuer,clusterissuer,certificates,orders,challenges > cert-manager-backup.yaml

Source

2.Delete:

# Uninstall the Helm chart
helm delete --purge <your release name here>

# Ensure the cert-manager CustomResourceDefinition resources do not exist:
kubectl delete crd \
    certificates.certmanager.k8s.io \
    issuers.certmanager.k8s.io \
    clusterissuers.certmanager.k8s.io

described in step 2 here

3.Install a fresh jetstack version:

# Install the CustomResourceDefinition resources separately
kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.9/deploy/manifests/00-crds.yaml

# Create the namespace for cert-manager
kubectl create namespace cert-manager

# Label the cert-manager namespace to disable resource validation
kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true

# Add the Jetstack Helm repository
helm repo add jetstack https://charts.jetstack.io

# Update your local Helm chart repository cache
helm repo update

# Install the cert-manager Helm chart
helm install --name <your release name here> --namespace cert-manager --version v0.9.1 jetstack/cert-manager

described here

4.Restore:

I have tried running

kubectl apply -f cert-manager-backup.yaml

as described here but this step actually didn't fully work for me. The Issuers were created (self signed and CA) but I could not re-create the Certificates and ClusterIssuer. These were the errors I have received:

Error from server (InternalError): Internal error occurred: failed calling webhook "clusterissuers.admission.certmanager.k8s.io": the server is currently unable to handle the request
Error from server (InternalError): Internal error occurred: failed calling webhook "certificates.admission.certmanager.k8s.io": the server is currently unable to handle the request

I had my original yaml files and was able to create the ClusterIssuer and Certificate by applying them

Answer 2

I can confirm that the above works. (@RVid answer)

Though, I've upgraded 0.5.0 to 0.9.1 and had to create a separate namespace to have 'no-downtime' upgrade.

#1 delete old CRDs
kubectl delete crd \
    certificates.certmanager.k8s.io \
    issuers.certmanager.k8s.io \
    clusterissuers.certmanager.k8s.io

#2 create SEPARATE namespace
$ kubectl create namespace cert-manager-new

#3 install new CRDs that corresponds to the new version of cert-manager

$ kubectl apply \
    -f https://raw.githubusercontent.com/jetstack/cert-manager/<VERSION>/deploy/manifests/00-crds.yaml


#4 ensure the NEW namespace has an additional label on it in order for the deployment to succeed
$ kubectl label namespace cert-manager-new certmanager.k8s.io/disable-validation="true"

#5 copy secrets to cert-manager-new namespace (For DNS, HTTP and Let's Encrypt account)

## Install the cert-manager helm chart
#  jetstack/cert-manager
$ helm install --name cert-manager-new --namespace cert-manager-new jetstack/cert-manager --values <your values file>

#6 apply ClusterIssuer with kubectl apply -f <file.yaml> 
Use config from: https://docs.cert-manager.io/en/latest/reference/issuers.html

The new instance of the cert manager will start synchronizing all the certificates you have without destroying the secrets. Eventually, all the certs will get renewed with new cert-manager.

Cheers.