After an OOM how can I detect the pod that caused it?

Question

I had some issues running some pods on a cluster, I want to know the way to detect which pod (and rc) is causing OOM on my nodes after the exception is thrown. I cannot access the node to check logs and kubectl describe node doesn't give me much information about this.

Thanks :)

Answer 1

Have you try running kubectl get events --watch to monitor the events on k8s and monitor the pod as well with kubectl logs -f podname

Answer 2

The only way I found to track what is happening is waiting before an OOM is thrown on the web UI link and tracking the pod's memory used (I had a node with 1.75 GB and a pod that was consuming 1.3GB). I tried to see the memory allocation with kubectl describe node [nodename] but I couldn't.