Loading a jar at runtime causes a NoClassDefFoundError/ClassNotFoundException

Question

Summary: Loading a jar from a running Java program causes a NoClassDefFoundError caused by a ClassNotFoundException caused by inter-class dependencies (e.g. import statements). How can I get around it?

The problem in more detail:

I am attempting to programmatically load a jar file -- let's call it "Server" -- into the Java Virtual Machine through my own Java program -- let's call it "ServerAPI" -- and use extension and some other tricks to modify the behavior of and interact with Server. ServerAPI depends on Server, but if Server is not present, ServerAPI still has to be able to run and download Server from a website.

To avoid errors caused by ServerAPI loading without satisfying its dependencies from Server, I have made a launcher -- let's call it "Launcher" -- that is intended to download Server and set up ServerAPI as necessary, then load Server and ServerAPI, then run ServerAPI.

However, when I attempt to load jars from Launcher, I get errors caused because the ClassLoaders are unable to resolve the other classes in the file that the class it's loading depends on. In short, if I try to load Class A, it will throw an error if A imports B because I haven't loaded B yet. However, if B also imports A, I'm stuck because I can't figure out how to load two classes at once or how to load a class without the JVM running its validation.

Why all the restrictions have led me to this problem:

I am attempting to modify and add to the behavior of Server, but for complicated legal reasons, I cannot modify the program directly, so I have created ServerAPI that depends on and can tweak the behavior of Server from the outside.

However, for more complicated legal reasons, Server and ServerAPI cannot simply be downloaded together. Launcher (see above) has to be downloaded with ServerAPI, then Launcher needs to download Server. Finally, ServerAPI can be run using Server as a dependency. That's why this problem is so complex.

This problem will also apply to a later part of the project, which will involve a plugin-based API interface that needs to be able to load and unload plugins from jar files while running.

Research I have already done on this problem:

I have read through and failed to be helped by:

• this question, which only addresses the issue of a single method and does not address inter-class dependency errors;
• this question, which will not work because I cannot shut down and restart the program every time a jar is loaded or unloaded (mainly for the plugin part I briefly mentioned);
• this question, which only works for situations where the dependencies are present when the program starts;
• this question, which has the same problem as #2;
• this question, which has the same problem as #3;
• this article, from which I learned about the hidden loadClass(String, boolean) method, but trying with true and false values did not help;
• this question, which has the same problem as #1;

and more. Nothing has worked.

//EDIT: Attempts I have made so far:

I have tried using URLClassLoaders to load the jar using the JarEntries from the JarFile similar to this question. I tried this both by using and calling a URLClassLoader's loadClass(String) method and by making a class that extends URLClassLoader so that I could utilize loadClass(String, boolean resolve) to try to force the ClassLoader to resolve all the classes it loads. Both ways, I got this same error:

I couldn't find the class in the JarEntry!
entry name="org/apache/logging/log4j/core/appender/db/jpa/converter/ContextMapAttributeConverter.class"
class name="org.apache.logging.log4j.core.appender.db.jpa.converter.ContextMapAttributeConverter"
java.lang.NoClassDefFoundError: javax/persistence/AttributeConverter
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at Corundum.launcher.CorundumClassLoader.load(CorundumClassLoader.java:52)
    at Corundum.launcher.CorundumLauncher.main(CorundumLauncher.java:47)
Caused by: java.lang.ClassNotFoundException: javax.persistence.AttributeConverter
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 12 more

//END EDIT

//EDIT 2:

Here is a sample of the code that I used to load a class while trying to resolve it. This was inside a class that I made that extends URLClassLoader. On the line beginning with Class<?> clazz = loadClass(, I have tried using true and false as the boolean argument; both attempts resulted in the same error above.

public boolean load(ClassLoadAction class_action, FinishLoadAction end_action) {
    // establish the jar associated with this ClassLoader as a JarFile
    JarFile jar;
    try {
        jar = new JarFile(jar_path);
    } catch (IOException exception) {
        System.out.println("There was a problem loading the " + jar_path + "!");
        exception.printStackTrace();
        return false;
    }

    // load each class in the JarFile through its JarEntries
    Enumeration<JarEntry> entries = jar.entries();

    if (entries.hasMoreElements())
        for (JarEntry entry = entries.nextElement(); entries.hasMoreElements(); entry = entries.nextElement())
            if (!entry.isDirectory() && entry.getName().endsWith(".class"))
                try {
                    /* this "true" in the line below is the whole reason this class is necessary; it makes the URLClassLoader this class extends "resolve" the class,
                     * meaning it also loads all the classes this class refers to */
                    Class<?> clazz = loadClass(entry.getName().substring(0, entry.getName().length() - 6).replaceAll("/", "."), true);
                    class_action.onClassLoad(this, jar, clazz, end_action);
                } catch (ClassNotFoundException | NoClassDefFoundError exception) {
                    try {
                        close();
                    } catch (IOException exception2) {
                        System.out.println("There was a problem closing the URLClassLoader after the following " + exception2.getClass().getSimpleName() + "!");
                        exception.printStackTrace();
                    }
                    try {
                        jar.close();
                    } catch (IOException exception2) {
                        System.out.println("There was a problem closing the JarFile after the following ClassNotFoundException!");
                        exception.printStackTrace();
                    }
                    System.out.println("I couldn't find the class in the JarEntry!\nentry name=\"" + entry.getName() + "\"\nclass name=\""
                            + entry.getName().substring(0, entry.getName().length() - 6).replaceAll("/", ".") + "\"");
                    exception.printStackTrace();
                    return false;
                }

    // once all the classes are loaded, close the ClassLoader and run the plugin's main class(es) load() method(s)
    try {
        jar.close();
    } catch (IOException exception) {
        System.out.println("I couldn't close the URLClassLoader used to load this jar file!\njar file=\"" + jar.getName() + "\"");
        exception.printStackTrace();
        return false;
    }

    end_action.onFinishLoad(this, null, class_action);
    System.out.println("loaded " + jar_path);
    // TODO TEST
    try {
        close();
    } catch (IOException exception) {
        System.out.println("I couldn't close the URLClassLoader used to load this jar file!\njar file=\"" + jar_path + "\"");
        exception.printStackTrace();
        return false;
    }
    return true;
}

//END EDIT 2

I realize that there must be a simple solution to this, but for the life of me I cannot seem to find it. Any help would make me eternally grateful. Thank you.

Answer 1

Embarassingly, I found that the answer was that the error message was telling the truth. javax.persistence.AttributeConverter, the class that the loader was claiming was not present, was not in the jar.

I fixed the issue by loading only the main class and the ClassLoader all references classes, essentially loading all the classes in the jar that are used in the program, which is all I need.

Now, I could have sworn that I checked for this before and found that class; I figure I must have actually checked the Apache open source repository for the class rather than the actual Server when I checked that. I can't remember.

In any case, AttributeConverter is missing. I don't know how or why they managed to compile a jar with missing dependencies, but I guess their main processes never use that part of the code, so it never threw errors.

I'm sorry to have wasted everyone's time...including my own. I have been stuck on this problem for a while now.

Moral of this story:

If you're trying to load an executable jar, don't bother loading all the classes in a jar unless you actually have to. Just load the main class; that will load everything the program needs to run.

//EDIT:

I have now started having the same error, but it does not appear until I attempt to call a method from a loaded class. The question is apparently still open. Please downvote and disregard this answer.