Menu
Please provide a simple step by step guide to looking into java heap dump from a Kubernetes pod.
561
If you want to get the heap dump, just follow the same steps you went through for the local environment. There is only one "but": the dump file is created inside your pod and not on your local machine. You need to copy the file first: kubectl cp <pod-name>:/dump. hprof ~/work/dump.
To get the process ID of either container, take note of the container ID or name, and use it in the following docker command: docker inspect --format '{{ . State. Pid }}' container-id-or-name.
and sometime it gets even harder to create the heap dump when we have resources request/limits specified on the container, the dump tool itself consume a lot resource at a peak and even exceed the resource limit threshold which trigger kubernetes restart that pod, therefore we could lose the heap dump.
Is there any other way to get the java heap dump from the pod? Most likely jmap is not installed in the container, so you can't use jmap in the container. It may just be a path problem, though I doubt it. I think jmap is in the JDK but not the JRE, and as a rule, I'd say there's no need for the full JDK in a containerized app.
Log in to the K8S and exec into the Pods where your java application is running. Copy the heap dump from pod to your local machine. Use any memory leak analysis tool. I'm using the Eclipse's Memory Analyzer plugin. You can check the number of objects and retained heap space.
The first part of the puzzle is that, when Kubernetes is killing your container because it is not responding to the health check, Kubernetes will just restart the container, but will not reschedule the pod, so it will not move it to another node.
kubectl exec -it herle-deployment-pod-5757d89d85-wrpc9 bash
get the process id (top command)
Create java heap dump
jmap -dump:live,format=b,file=<file_name>.bin <process_id>
Example:
jmap -dump:live,format=b,file=application_heap_dump.bin 1
kubectl cp <pod_name>:<heap_file> <your local destination directory>
Example:
kubectl cp herle-deployment-pod-5757d89d85-wrpc9:/tmp/application_heap_dump.bin /Users/amritharajherle/Desktop/application_heap_dump.bin
67
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With