Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Java - Relocate references at runtime

So first off I know that you can relocate all references your compiled jar with various shadow plugins on different build systems. I know how that works and am already using it. However I ran into a problem where I can't do that at compile time.

I'll simplify my situation so it's easier to understand (but I'll explaining the full picture at the bottom, in case you are curious).
I'm writing a plugin for two different (but similar) systems (one jar forall). Those platforms are in charge of starting the underlying software and loading/starting all plugins (so I don't have control over the application, including start parameters).
Platform A offers me a library (let's call it com.example.lib). And so does platform B. But it decided to relocate it to org.b.shadow.com.example.lib.
Now in the core code (the code used on both platforms) of my plugin I use the library. Now while I can detect on which platform I am on, I currently do not know how I can rewrite all references in my code to the library at runtime so it works on platform B.

From what I've found it seems like I need to use a custom ClassLoader to achieve that. The issue here being that I don't know I could make the runtime use my custom ClassLoader. Or where to start really.
One important thing is that those relocations may only affect references in classes from my packages (me.brainstone.project for example).
Another dependency I use (and have shaded in) uses ASM and ASM Commons, so if it is possible doing it with those, that would be amazing!

So in summary. I would like to optionally relocate references (to other classes) in only my classes at runtime.

Edit:

While throught my entire (orginal) post I only ever talked about one library, I would like to point out that I will be doing this for serveral libaries. And there for doing things that require me to put a significant effort (writing wrappers for every library (class or section) would be consider as a significant effort) into allowing to me use a libary is not what I am looking for. Instead I want a solution that requries minimal additons for adding new libraries into the mix.


Now here is a a bit more detailed explanation of my setup.
Fist I'd like to preface that I am aware that I can just create two different jars for the different platforms. And I am already doing that. But since surprisingly many people can't seem to figure that out and I'm getting tired of explaining it over and over again (those are the people that wouldn't read docs to save their lives) I'd like to just offer a single jar for both, even if it means I need to spend a significant time on getting it to work (I much prefer this over constantly explaining it).
Now my actual setup looks like this: On platform A the library is provided but on platform B it isn't. I know that other plugins often use the library by shading it in (many not relocating causing all kinds of issues). So to prevent any conflicts I download the library, relocate the classes inside that jar with jar-relocator and then inject it into the classpath using reflections. Downside in this case I currently cannot use the library if it's relocated. That's why I'd like to change the references in my code at runtime. And it also explains why I don't want to change the references of other classes, because I don't want to accidentally break those other plugins. I also think that if I can somehow use my own ClassLoader that I don't need to inject jars into the main ClassLoader because then I can just tell that ClassLoader to use the additional jars without having to resort to reflections.
But as I said, from what I understand the problem is the same as in the simplified version.

like image 469
BrainStone Avatar asked Aug 28 '19 20:08

BrainStone


1 Answers

First you should think about different solution, as every other solution is better than this one, so possible ones:

  1. Just create separate modules.
  2. Use some code generation at compile time to generate that modules so you don't need to duplicate your code, look at https://github.com/vigna/fastutil for example.

But if you really want to do it in very dirty way:
Use java agents. This require use of jdk jvm or/and additional startup arguments. You should probably use byte-buddy-agent library if you want to do this at runtime without startup arguments, and there is also dirty trick on java 8 to run agents in runtime even without proper files from jdk - by just injecting them manually, probably also possible on java 9+ but so far I didn't have time and need to find a way to do this. You can see my instructions here https://github.com/raphw/byte-buddy/issues/374#issuecomment-343786107
But if possible the best way is to just use command line argument to attach agent .jar as separate thing.
First thing to do is to write a class file transformer that will do all the logic you need:

public class DynamicLibraryReferenceTransformer implements ClassFileTransformer {
    private final String packageToProcess;
    private final String originalPackage;
    private final String resolvedPackage;

    DynamicLibraryReferenceTransformer(String packageToProcess, String originalPackage, String resolvedPackage) {
        this.packageToProcess = packageToProcess;
        this.originalPackage = originalPackage;
        this.resolvedPackage = resolvedPackage;
    }

    @Override
    public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain,
                            byte[] classfileBuffer) {
        if (! className.startsWith(this.packageToProcess)) {
            return null; // return null if you don't want to perform any changes
        }
        Remapper remapper = new Remapper() {
            @Override
            public String map(String typeName) {
                return typeName.replace(originalPackage, resolvedPackage);
            }
        };
        ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
        ClassRemapper classRemapper = new ClassRemapper(cw, remapper);
        ClassReader classReader = new ClassReader(classfileBuffer);
        classReader.accept(classRemapper, 0);
        return cw.toByteArray();
    }
}

And then you just need to apply it as java agent, either in runtime:

static { 
    Instrumentation instrumentation= ByteBuddyAgent.install();
    // note that this uses internal names, with / instead of dots, as I'm using simple .replace it's good idea to keep that last / to prevent conflicts between libraries using similar packages. (like com/assist vs com/assistance)
    instrumentation.addTransformer(new DynamicLibraryReferenceTransformer("my/pckg/", "original/pckg/", "relocated/lib/"), true);
    // you can retransform existing classes if needed but I don't suggest doing it. Only needed if some classes you might need to transform are already loaded
    // (classes are loaded on first use, with some smaller exceptions, like return types of methods of loaded class are also loaded if I remember correctly, where fields are not)
    // you can also just retransform only known classes
    instrumentation.retransformClasses(install.getAllLoadedClasses());
}

This code should be run as quickly as possible, like in static block of code inside your main class.

Better option is to include agent to JVM at startup using command line:

First you will need to create new project as this will be separate .jar, and create manifest with Premain-Class: mypckg.AgentMainClass that you will include in meta-inf of agent .jar.
Use same transformer as above, and then you just need to write very simple agent like this:

public class AgentMainClass {
    public static void premain(String agentArgs, Instrumentation instrumentation) {
        instrumentation.addTransformer(new DynamicLibraryReferenceTransformer("my/pckg/", "original/pckg/", "relocated/lib/"), true);
    }
}

And now just include it in your java command to run the app (or server probably) -javaagent:MyAgent.jar.
Note that you can include code of agent and manifest inside your main (plugin?) .jar, just be sure to not mix up dependencies, classes for agent will be loaded using different class loader, so don't make calls between app and agent, that will be 2 separate things inside single .jar.

This uses org.ow2.asm.asm-all library, and net.bytebuddy.byte-buddy-agent (only for runtime version) library.

like image 54
GotoFinal Avatar answered Oct 04 '22 16:10

GotoFinal