Profiling jetty with visualvm is super slow

Question

I have a wicket+spring+hibernate application running on Jetty. When I start CPU profiling it with VisualVM (jdk 1.7.0_9) it first stalls for several minutes with console prints:

Profiler Agent: 250 classes cached.
Profiler Agent: 250 classes cached.

These lines are repeated around 20 times, then VisualVM says it has started instumentation and instrumented around 8000 methods.

Now after this I click a button on my web application and again the application completely hangs for few minutes while console prints out lines like:

Profiler Agent: Redefining 100 classes at idx 100, out of total 336 

After this I get profiling results but they are pretty useless as almost 99.6% of the time is spent by

sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run()

This makes VisualVM totally unusuable. Any guesses what could be the culprit here?

I'm running Jetty 8.1.2.v20120308

Answer 1

The answer is to narrow the scope of what classes are being instrumented.

Click on the settings option in the profiler and look at "Do not profile classes" or "profile only classes". Be sure to exclude third party libraries that you don't want to examine. For example, I was using Jython in my app and the profiler was trying to instrument thousands of classes, likely including classes dynamically generated at runtime (not good).

Answer 2

I would suggest to start with CPU sampling first. Once you have an idea what is wrong, you can switch to profiling to get detailed information. Be sure to read Profiling With VisualVM, Part 1 and Profiling With VisualVM, Part 2 to get more information how to set up profiling of your Jetty server.