Menu
When a Pod is evicted I'm supposed to find messages about the eviction at /dev/termination-log (or other path defined in terminationMessagePath property) inside the container filesystem.
I'm trying to track OOMKilled events, but after OOMKilled occured and the Pod was restarted, the /dev/termination-log is empty.
I tried to set terminationMessagePolicy property to FallbackToLogsOnError hoping to find out messages in container log and I did not find anything about the OOMKilled there.
By executing kubectl describe pod podname I can find out that the termination reason was OOMKilled, but I would like to be able to follow this events in order to integrate it with fluentd and logstash.
Our Kubernetes cluster version is v1.9.0 and it's running on-premise. The installation was done using kubeadm on top of CentOS 7.
731
To get Kubectl pod logs, you can access them by adding the -p flag. Kubectl will then get all of the logs stored for the pod. This includes lines that were emitted by containers that were terminated.
If the Container continues to consume memory beyond its limit, the Container is terminated. If a terminated Container can be restarted, the kubelet restarts it, as with any other type of runtime failure.
You can ssh to the machine the pod was running and exec journalctl -u kubelet
190
The way I'm using - is looking to dmesg on nodes and track events from there
34
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
