How to retransform an executing method with JVMTI agent which has no further invocations?

Question

I am instrumenting a class file during runtime for various purposes. I'm using a JVMTI agent to that end. My strategy to instrument a method is to call RetransformClasses function to invoke ClassFileLoadHook. This strategy works fine for all the methods which have any further invocation after the time of instrumentation because actual instrumentation happens at subsequent function call, but it doesn't work for any method which does not have further invocations like main function in a program.

I want to instrument a method on the fly during its execution. I want some procedure like On-Stack Replacement (OSR) of the instrumented code. Is there any strategy available in JVMTI or any other approach????

PS: I'm open to editing/patching OpenJDK source code if that can help.

Answer 1

After some further thinking, I believe you are ask for something that might maybe (maybe!) be possible technically; but require lots of efforts; but conceptually it is not a good approach.

I assume your requirement is actually that you want to instrument any kind of application thrown at you in order to improve its performance by doing "under the cover parallelizing".

So, instead of having a real solution, I mainly have a list of concerns:

• First of all, if you even want to modify methods that were already triggered and that are currently executed, you are not only talking about instrumenting. What you actually want to do is to provide your own "JIT" mechanism - while the JVM JIT is also there, and doing its job.
• So, if you are really serious about this; and want to make sure that even the things in any main() can benefit from your optimizations - then I think, conceptually, you are better of designing and implementing your own JVM then.
• Then I am wondering: you say want to cover main() methods that are already running "long time loops". That sounds like you intend to fix bad designs by throwing your instrumentation at it. I think the more sane approach is: look into such applications, and improve their design.
• In the sense of: if "parallelizing" arbitrary applications would be "that easy" - it would be part of the JVM anyway. And the fact that it is not; and that the JVM doesn't do such kind of optimizations is for a good reason: it is probably super hard to get that correct and robust.

In other words: I guess you have an XY problem; and the X problem is that the application(s) you are dealing could benefit from "parallelizing". But that is something that is very hard to do "in general".

In that sense; I would rather define some kind of architecture (that might include specific, well-defined steps how an application should "start up"; so that your instrumentation can do its work successfully) and gain experience with that approach first. Meaning: tell your folks to not put "long running loops" into their main() in the first place (as said; that alone sounds like pretty bad design to me!).