How to generate a cpu snapshot in production?

Question

With VisualVM, I can attach to a local java process, start/stop CPU profiling, and generate a snapshot. I don't even need to instrument the process with any special agents etc.

Is it possible to do the same thing on a headless production server? I know I can configure the process to attach the debugger remotely, but I want to minimize the amount and impact of changing the customer environment. I just want to run a command to profile for a while and generate a snapshot file, which I can scp to my workstation for analysis with VisualVM or equivalent.

Preferably, the whatever program I run to do this should be freely redistributable, so that I can bundle it with my Java app and have it be available in the customer environment.

Ideally, just like with VisualVM, I'd like to be able to attach to a running system without a restart.

The closest I've found so far is instrumenting with hprof:

  java -agentlib:hprof=cpu=samples,interval=20,depth=3 classname

but that requires a restart to start profiling and another to stop, as I don't see a way to turn sampling on and off at runtime.

Is there a better way? If there's no out-of-the-box solution, how would one build it? It must be possible since VisualVM does it.

Answer 1

You don’t need a debugging connection, you only need a JMX connection. But maybe that’s what you meant.

To recall, when you run your application with the appropriate options like

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9876
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false

(replace the latter two if you need secure transport) you can just connect with JVisualVM from another machine with GUI using the “Add JMX connection” and servername:9876 as the address.

Details are given in http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html and http://visualvm.java.net/jmx_connections.html

---

When you are using JVisualVM to connect to a local JVM, the only difference is that it will detect local applications and tell them to load the JMX agent if it is not already running.

So you can write a small local command line application to be executed on the server and connect to the JVM and start the JMX agent. Once the JMX agent is started you can connect with JVisualVM remotely. The program in this answer shows how to do this.

It uses the Attach API to locate the JVM and tell it to load the JMX agent and opens the JMX connection. The second step can be omitted if you will open the JMX connection using JVisualVM remotely.

Alternatively you can extend this program to a command line sampling tool using the opened JMX connection. The first example code of this answer gives an idea of how to implement a sampler. It samples its own JVM but since it uses the ThreadMXBean you can change it to work for remote connections as well easily.