Why am I getting a NoClassDefFoundError exception rather than a StackOverflow error?

Question

Playing with Java (v9 specifically) I found this situation:

import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

interface A {
    static A staticMethod() {
        try {
            Method method = A.class.getDeclaredMethods()[0];
            return (A) method.invoke(null);
        } catch (Exception e) {
            e.printStackTrace();
        }

        return null;
    }
}

public class Test {
    public static void main(String[] args) {
        A.staticMethod();
    }
}

That program flow should cause a StackOverflow error, however, I'm getting a NoClassDefFoundError.

*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at JPLISAgent.c line: 880
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at JPLISAgent.c line: 880
*** java.lang.instrument ASSERTION FAILED ***: "!errorOutstanding" with message transform method call failed at JPLISAgent.c line: 880
Exception in thread "main" 
Exception: java.lang.NoClassDefFoundError thrown from the UncaughtExceptionHandler in thread "main"

According to Javadoc

Class NoClassDefFoundError

Thrown if the Java Virtual Machine or a ClassLoader instance tries to load in the definition of a class (as part of a normal method call or as part of creating a new instance using the new expression) and no definition of the class could be found.

The searched-for class definition existed when the currently executing class was compiled, but the definition can no longer be found.

This is a weird error message, is it a bug?

UPDATE: Bug Report Id: 9052375

Executed from the command line and prints the expected error: The problem was, the exceptions used in catch.

Answer 1

This is not a bug and it also has nothing to do with static methods in interfaces.

The java.lang.instrument ASSERTION FAILED message is also not relevant and is just an artifact of running the code from IDE. Running the same class from the command line will result in Exception in thread "main" only.

Lets simplify your example to

public class Test {
    public static void main( String[] args ) throws Exception {
        recursive();
    }

    public static void recursive() throws Exception {
        try {
            Test.class
                    .getDeclaredMethod( "recursive" )
                    .invoke( null );
        } catch ( InvocationTargetException e ) {
            e.printStackTrace();
        }
    }
}

What is going on:

• The recursive method causes StackOverflowError, as expected.
• The StackOverflowError is wrapped into InvocationTargetException which is thrown from the deepest nested call to method.invoke().
• The InvocationTargetException is immediately caught and JVM tries to execute printStackTrace() but in order to do that it needs to load some classes. But remember that at this point the stack is depleted and any non-trivial methods will hit StackOverflowError again, which is exactly what happens somewhere inside the class loader when it tries to load some class required to print a stack trace. The class loader did found the class, but failed to load and initialize it, and it reports that as NoClassDefFoundError.

The following code will prove that InvocationTargetException indeed wraps StackOverflowError:

public class Test {
    public static void main( String[] args ) throws Exception {
        recursive();
    }

    public static void recursive() throws Exception {
        try {
            Test.class
                    .getDeclaredMethod( "recursive" )
                    .invoke( null );
        } catch ( InvocationTargetException e ) {
            System.out.println(e);
            System.out.println(e.getTargetException());
        }
    }
}

And the following code will prove that if classes required to execute printStackTrace() are already loaded, then the code behaves as expected (prints a stack trace for InvocationTargetException caused by StackOverflowError:

public class Test {
    public static void main( String[] args ) throws Exception {
        new Exception().printStackTrace(); // initialize all required classes
        recursive();
    }

    public static void recursive() throws Exception {
        try {
            Test.class
                    .getDeclaredMethod( "recursive" )
                    .invoke( null );
        } catch ( InvocationTargetException e ) {
            e.printStackTrace();
        }
    }
}

The open question is why reflection API handlesStackOverflowError at all, instead of simply terminating the whole call chain with the error.