Lambda expressions and anonymous classes don't work when loaded as hidden classes

I am trying to compile and load dynamically generated Java code during runtime. Since both ClassLoader::defineClass and Unsafe::defineAnonymousClass have serious drawbacks in this scenario, I tried using hidden classes via Lookup::defineHiddenClass instead. This works fine for all classes that I tried to load, except for those that call lambda expressions or contain anonymous classes.

Calling a lambda expression throws the following exception:

Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
    at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
    ... 1 more

Executing code that instantiates an anonymous class throws the following error:

Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
  Location:
    tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
  Reason:
    Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
  Current Frame:
    bci: @5
    flags: { }
    locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
    stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
  Bytecode:
    0000000: bb00 1159 2ab7 0013 4cb1               

    at java.base/java.lang.ClassLoader.defineClass0(Native Method)
    at java.base/java.lang.System$2.defineClass(System.java:2193)
    at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
    at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
    at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
    at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)

This is a short example that recreates the problem:

import java.lang.invoke.MethodHandles;

public class HiddenClassLambdaTest {
    /** This class is to be loaded and executed as hidden class */
    public static final class LambdaRunner implements Runnable {
        @Override public void run() {
            Runnable runnable = () -> System.out.println("Success");
            runnable.run();
        }
    }
    
    public static void main(String[] args) throws Throwable {
        // Path to the class file of the nested class defined above
        String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
        // Class file content of the LambdaRunner class
        byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
        Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
        Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
        lambdaRunnerInstance.run();
    }
}

I've already tried compiling and running the code with different JDKs, using different ways to create new instances of the hidden class, searching for bugs at https://bugs.openjdk.java.net/, messing with the bytecode itself and several other things. I am not an expert on Java internals, so I am not sure whether I have not understood the JEP that introduced hidden classes correctly.

Am I doing something wrong, is this just impossible or is this a bug?

Edit: The JEP states

Migration should take the following into account: To invoke private nestmate instance methods from code in a hidden class, use invokevirtual or invokeinterface instead of invokespecial. Generated bytecode that uses invokespecial to invoke a private nestmate instance method will fail verification. invokespecial should only be used to invoke private nestmate constructors.

This might be the problem for the anonymous class. Is there a way to compile the code such that invokespecial is avoided in the bytecode?

You can not turn arbitrary classes into hidden classes.

The documentation of defineHiddenClass contains the sentence

• On any attempt to resolve the entry in the run-time constant pool indicated by this_class, the symbolic reference is considered to be resolved to C and resolution always succeeds immediately.

What it doesn’t spell out explicitly is that this is the only place where a type resolution ever ends up at the hidden class.

But it has been said unambiguously in bug report JDK-8222730:

For a hidden class, its specified hidden name should only be accessible through the hidden class's 'this_class' constant pool entry.

The class should not be accessible by specifying its original name in, for example, a method or field signature even within the hidden class.

Which we can check. Even a simple case like

public class HiddenClassLambdaTest {

    public static void main(String[] args) throws Throwable {
        byte[] classFileContents = HiddenClassLambdaTest.class
            .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
            .readAllBytes();
        var hidden = MethodHandles.lookup()
            .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
        Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
            hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
        lambdaRunnerInstance.run();
    }

    static class LambdaRunner implements Runnable {
        LambdaRunner field = this;

        @Override
        public void run() {
        }
    }
}

will already fail. Note that it is a special case that the attempt to resolve the original class name LambdaRunner within the hidden class will not fail, as you used an existing class as template. So you get an IncompatibleClassChangeError or a VerifierError due to mismatches between the hidden class and the existing LambdaRunner class. When you don’t use a class definition of an existing class, you’d get a NoClassDefFoundError.

The same applies to

    static class LambdaRunner implements Runnable {
        static void method(LambdaRunner arg) {
        }

        @Override
        public void run() {
            method(this);
        }
    }

As the cited bug report said, neither field nor methods can refer to the hidden class in their signature.

A less intuitive example is

    static class LambdaRunner implements Runnable {
        @Override
        public void run() {
            System.out.println("" + this);
        }
    }

which will fail depending on the compiler and options, as when the StringConcatFactory is used, the behavior is like an invocation of a method having all non-constant parts as parameters and returning a String. So this is another case of having the hidden class in a method signature.

---

Lambda expressions are special, as a class like

    static class LambdaRunner implements Runnable {
        @Override
        public void run() {
            Runnable runnable = () -> System.out.println("Success");
            runnable.run();
        }
    }

gets compiled similar to

    static class LambdaRunner implements Runnable {
        @Override
        public void run() {
            Runnable runnable = LambdaRunner::lambdaBody;
            runnable.run();
        }
        private static void lambdaBody() {
            System.out.println("Success");
        }
    }

which doesn’t have the hidden class in the method signature, but has to refer to the method holding the body of the lambda expression as a MethodReference. Within the constant pool, the description of this method refers to its declaring class using the this_class entry. So it gets redirected to the hidden class as described in the documentation.

But the construction of the MethodType as part of the MethodReference does not use this information to load a Class like a class literal would do. Instead, it tries to load the hidden class through the defining class loader, which fails with the NoClassDefFoundError you have posted.

This seems to be related to JDK-8130087 which suggests that ordinary method resolution differs from the way, MethodType works, which can make MethodType fail where just invoking the method would work.

But it’s possible to demonstrate that even fixing this issue wouldn’t solve the general problem:

    static class LambdaRunner implements Runnable {
        @Override
        public void run() {
            var lookup = MethodHandles.lookup();
            var noArgVoid = MethodType.methodType(void.class);
            try {
                MethodHandle mh = LambdaMetafactory.metafactory(lookup, "run",
                    MethodType.methodType(Runnable.class), noArgVoid,
                    lookup.findStatic(LambdaRunner.class, "lambdaBody", noArgVoid),
                    noArgVoid).getTarget();
                System.out.println("got factory");
                Runnable runnable = (Runnable)mh.invokeExact();
                System.out.println("got runnable");
                runnable.run();
            }
            catch(RuntimeException|Error e) {
                throw e;
            }
            catch(Throwable e) {
                throw new AssertionError(e);
            }
        }
        private static void lambdaBody() {
            System.out.println("Success");
        }
    }

This bypasses the problem described above and calls the LambdaMetafactory manually. When being redefined as hidden class, it will print:

got factory
got runnable
Exception in thread "main" java.lang.NoClassDefFoundError: test/HiddenClassLambdaTest$LambdaRunner/0x0000000800c01400
    at test/test.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:15)
Caused by: java.lang.ClassNotFoundException: test.HiddenClassLambdaTest$LambdaRunner.0x0000000800c01400
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    ... 1 more

which shows that all obstacles have been circumvented, but when it comes to the actual invocation from the generated Runnable to the method holding the lambda body, it will fail due to the fact that the target class is hidden. A JVM with eager resolution of symbolic references might fail earlier, i.e. the example might not print got runnable then.

Unlike the old JVM anonymous classes, there is no way to link to a hidden class, not even from another hidden class.

---

The bottom line is, as said at the beginning, you can not turn arbitrary classes into hidden classes. Lambda expressions are not the only feature not working with hidden classes. It’s not a good idea to try and get surprised. Hidden classes should only be used in conjunction with bytecode generators carefully using only features known to work.