Menu
I am using cert-manager 0.5.2 to manage Let's Encrypt certificates on our Kubernetes cluster.
I was using the Let's Encrypt staging environment, but have now moved to use their production certificates. The problem is that my applications aren't updating to the new, valid certificates.
I must have screwed something up while updating the issuer, certificate, and ingress resources, but I can't see what. I have also reinstalled the NGINX ingress controller and cert-manager, and recreated my applications, but I am still getting old certificates. What can I do next?
Describing the letsencrypt cluster issuer:
Name: letsencrypt
Namespace:
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"certmanager.k8s.io/v1alpha1","kind":"ClusterIssuer","metadata":{"annotations":{},"name":"letsencrypt","namespace":""},"spec":{"acme":{"e...
API Version: certmanager.k8s.io/v1alpha1
Kind: ClusterIssuer
Metadata:
Cluster Name:
Creation Timestamp: 2019-01-04T09:27:49Z
Generation: 0
Resource Version: 130088
Self Link: /apis/certmanager.k8s.io/v1alpha1/letsencrypt
UID: 00f0ea0f-1003-11e9-997f-ssh3b4bcc625
Spec:
Acme:
Email: [email protected]
Http 01:
Private Key Secret Ref:
Key:
Name: letsencrypt
Server: https://acme-v02.api.letsencrypt.org/directory
Status:
Acme:
Uri: https://acme-v02.api.letsencrypt.org/acme/acct/48899673
Conditions:
Last Transition Time: 2019-01-04T09:28:33Z
Message: The ACME account was registered with the ACME server
Reason: ACMEAccountRegistered
Status: True
Type: Ready
Events: <none>
Describing the tls-secret certificate:
Name: tls-secret
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"certmanager.k8s.io/v1alpha1","kind":"Certificate","metadata":{"annotations":{},"name":"tls-secret","namespace":"default"},"spec":{"acme"...
API Version: certmanager.k8s.io/v1alpha1
Kind: Certificate
Metadata:
Cluster Name:
Creation Timestamp: 2019-01-04T09:28:13Z
Resource Version: 130060
Self Link: /apis/certmanager.k8s.io/v1alpha1/namespaces/default/certificates/tls-secret
UID: 0f38w7y4-1003-11e9-997f-e6e9b4bcc625
Spec:
Acme:
Config:
Domains:
mydomain.com
Http 01:
Ingress Class: nginx
Dns Names:
mydomain.com
Issuer Ref:
Kind: ClusterIssuer
Name: letsencrypt
Secret Name: tls-secret
Events: <none>
Describing the aks-ingress ingress controller:
Name: aks-ingress
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
TLS:
tls-secret terminates mydomain.com
Rules:
Host Path Backends
---- ---- --------
mydomain.com
/ myapplication:80 (<none>)
Annotations:
kubectl.kubernetes.io/last-applied-configuration: ...
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
certmanager.k8s.io/cluster-issuer: letsencrypt
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 21m nginx-ingress-controller Ingress default/aks-ingress
Normal CREATE 21m nginx-ingress-controller Ingress default/aks-ingress
Logs for cert-manager after restarting the server:
I0104 09:28:38.378953 1 setup.go:144] Skipping re-verifying ACME account as cached registration details look sufficient.
I0104 09:28:38.379058 1 controller.go:154] clusterissuers controller: Finished processing work item "letsencrypt"
I0104 09:28:38.378953 1 setup.go:144] Skipping re-verifying ACME account as cached registration details look sufficient.
I0104 09:28:38.379058 1 controller.go:154] clusterissuers controller: Finished processing work item "letsencrypt"
I0104 09:28:38.378455 1 controller.go:140] clusterissuers controller: syncing item 'letsencrypt'
I0104 09:28:38.378455 1 controller.go:140] clusterissuers controller: syncing item 'letsencrypt'
I0104 09:28:33.440466 1 controller.go:185] certificates controller: Finished processing work item "default/tls-secret"
I0104 09:28:33.440417 1 sync.go:206] Certificate default/tls-secret scheduled for renewal in 1423 hours
I0104 09:28:33.440466 1 controller.go:185] certificates controller: Finished processing work item "default/tls-secret"
I0104 09:28:33.440417 1 sync.go:206] Certificate default/tls-secret scheduled for renewal in 1423 hours
I0104 09:28:33.439824 1 controller.go:171] certificates controller: syncing item 'default/tls-secret'
I0104 09:28:33.439824 1 controller.go:171] certificates controller: syncing item 'default/tls-secret'
I0104 09:28:33.377556 1 controller.go:154] clusterissuers controller: Finished processing work item "letsencrypt"
I0104 09:28:33.377556 1 controller.go:154] clusterissuers controller: Finished processing work item "letsencrypt"
I0104 09:28:33.359246 1 helpers.go:147] Setting lastTransitionTime for ClusterIssuer "letsencrypt" condition "Ready" to 2019-01-04 09:28:33.359214315 +0000 UTC m=+79.014291591
I0104 09:28:33.359178 1 setup.go:181] letsencrypt: verified existing registration with ACME server
I0104 09:28:33.359178 1 setup.go:181] letsencrypt: verified existing registration with ACME server
I0104 09:28:33.359246 1 helpers.go:147] Setting lastTransitionTime for ClusterIssuer "letsencrypt" condition "Ready" to 2019-01-04 09:28:33.359214315 +0000 UTC m=+79.014291591
I0104 09:28:32.427832 1 controller.go:140] clusterissuers controller: syncing item 'letsencrypt'
I0104 09:28:32.427978 1 controller.go:182] ingress-shim controller: Finished processing work item "default/aks-ingress"
I0104 09:28:32.427832 1 controller.go:140] clusterissuers controller: syncing item 'letsencrypt'
I0104 09:28:32.427832 1 controller.go:168] ingress-shim controller: syncing item 'default/aks-ingress'
I0104 09:28:32.428133 1 logger.go:88] Calling GetAccount
I0104 09:28:32.427936 1 sync.go:140] Certificate "tls-secret" for ingress "aks-ingress" already exists
I0104 09:28:32.427965 1 sync.go:143] Certificate "tls-secret" for ingress "aks-ingress" is up to date
I0104 09:28:32.427978 1 controller.go:182] ingress-shim controller: Finished processing work item "default/aks-ingress"
I0104 09:28:32.428133 1 logger.go:88] Calling GetAccount
I0104 09:28:32.427936 1 sync.go:140] Certificate "tls-secret" for ingress "aks-ingress" already exists
I0104 09:28:32.427832 1 controller.go:168] ingress-shim controller: syncing item 'default/aks-ingress'
I0104 09:28:32.427965 1 sync.go:143] Certificate "tls-secret" for ingress "aks-ingress" is up to date
I0104 09:28:29.439299 1 controller.go:171] certificates controller: syncing item 'default/tls-secret'
E0104 09:28:29.439586 1 controller.go:180] certificates controller: Re-queuing item "default/tls-secret" due to error processing: Issuer letsencrypt not ready
I0104 09:28:29.439404 1 sync.go:120] Issuer letsencrypt not ready
E0104 09:28:29.439586 1 controller.go:180] certificates controller: Re-queuing item "default/tls-secret" due to error processing: Issuer letsencrypt not ready
I0104 09:28:29.439299 1 controller.go:171] certificates controller: syncing item 'default/tls-secret'
I0104 09:28:29.439404 1 sync.go:120] Issuer letsencrypt not ready
I0104 09:28:27.404656 1 controller.go:68] Starting certificates controller
I0104 09:28:27.404606 1 controller.go:68] Starting issuers controller
I0104 09:28:27.404325 1 controller.go:68] Starting ingress-shim controller
I0104 09:28:27.404606 1 controller.go:68] Starting issuers controller
I0104 09:28:27.404325 1 controller.go:68] Starting ingress-shim controller
I0104 09:28:27.404269 1 controller.go:68] Starting clusterissuers controller
I0104 09:28:27.404656 1 controller.go:68] Starting certificates controller
I0104 09:28:27.404269 1 controller.go:68] Starting clusterissuers controller
I0104 09:28:27.402806 1 leaderelection.go:184] successfully acquired lease kube-system/cert-manager-controller
I0104 09:28:27.402806 1 leaderelection.go:184] successfully acquired lease kube-system/cert-manager-controller
I0104 09:27:14.359634 1 server.go:84] Listening on http://0.0.0.0:9402
I0104 09:27:14.357610 1 controller.go:126] Using the following nameservers for DNS01 checks: [10.0.0.10:53]
I0104 09:27:14.357610 1 controller.go:126] Using the following nameservers for DNS01 checks: [10.0.0.10:53]
I0104 09:27:14.358408 1 leaderelection.go:175] attempting to acquire leader lease kube-system/cert-manager-controller...
I0104 09:27:14.359634 1 server.go:84] Listening on http://0.0.0.0:9402
I0104 09:27:14.356692 1 start.go:79] starting cert-manager v0.5.2 (revision 9e8c3ad899c5aafaa360ca947eac7f5ba6301035)
I0104 09:27:14.358408 1 leaderelection.go:175] attempting to acquire leader lease kube-system/cert-manager-controller...
I0104 09:27:14.356692 1 start.go:79] starting cert-manager v0.5.2 (revision 9e8c3ad899c5aafaa360ca947eac7f5ba6301035)
Certificate resource:
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: tls-secret
spec:
secretName: tls-secret
dnsNames:
- mydomain.com
acme:
config:
- http01:
ingressClass: nginx
domains:
- mydomain.com
issuerRef:
name: letsencrypt
kind: ClusterIssuer
713
You can renew your certificates manually at any time with the kubeadm certs renew command. This command performs the renewal using CA (or front-proxy-CA) certificate and key stored in /etc/kubernetes/pki . After running the command you should restart the control plane Pods.
cert-manager will automatically renew Certificate s. It will calculate when to renew a Certificate based on the issued X. 509 certificate's duration and a 'renewBefore' value which specifies how long before expiry a certificate should be renewed.
Kubernetes-internal certificates expire after one year. If you do not renew your certificate, Sisense ceases to function and you get the following error: "Part of the existing bootstrap client certificate expired". You have several options for renewing your certificate.
cert-manager adds certificates and certificate issuers as resource types in Kubernetes clusters, and simplifies the process of obtaining, renewing and using those certificates. It can issue certificates from a variety of supported sources, including Let's Encrypt, HashiCorp Vault, and Venafi as well as private PKI.
in this case the problem went away after recreating the secret and the cert-manager certificate resource.
generally what you want to check, annotations on your ingress resource (certmanager.k8s.io/cluster-issuer: letsencrypt), cert-manager certificate resource, ssl certificate secret in k8s and in ingress resource
83
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
