Is it possible to somehow send alerts (to email / slack) based on events that occur within a Kubernetes cluster?
In particular, it would be useful to get an alert if a pod has restarted unexpectedly or if a pod cannot start. Similarly it would be useful to know if a pod's CPU usage was over a certain threshold and get an alert.
We have Heapster (with InfluxDB / Grafana backend) installed. While this gives useful data, it unfortunately does not provide us with alerting.
Accessing Kubernetes Events Running the kubectl describe command on specific cluster resources will list the events for that resource. A more generic way of doing this is by running the kubectl get events command, which lists the specific resources' events or the entire cluster.
Access event logs Developers, application and infrastructure operators can use the kubectl describe command against specific resources, or use the more generic kubectl get event command to list events for a specific resource, or for the entire cluster.
A Kubernetes event is an object that shows what's happening inside a cluster, node, pod, or container. These objects are usually generated in response to changes that occur inside your K8s system. The Kubernetes API Server enables all core components to create these events.
Both sysdig and Datadog provide this functionality as well.
You can check and probably replace InfluxDB with http://prometheus.io (I am not sure about your cases but generally it is possible) which has built-in metrics support.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With