Why does LambdaMetafactory fail when using a custom functional interface (but Function works fine)?

Given:

import java.lang.invoke.LambdaMetafactory;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.MethodHandles;
import java.lang.invoke.MethodType;
import java.util.function.Function;

class Testcase
{
    @FunctionalInterface
    public interface MyBuilder1<R>
    {
        R apply(String message);
    }

    @FunctionalInterface
    public interface MyBuilder2<R>
    {
        R apply(Object message);
    }

    public static void main(String[] args) throws Throwable
    {
        Class<?> clazz = IllegalArgumentException.class;

        MethodHandles.Lookup lookup = MethodHandles.lookup();
        MethodHandle mh = lookup.findConstructor(clazz, MethodType.methodType(void.class, String.class));
        MethodHandle myFunctionConstructor = LambdaMetafactory.metafactory(
            lookup,
            "apply",
            MethodType.methodType(Function.class),
            mh.type().erase(),
            mh,
            mh.type()
        ).getTarget();

        MethodHandle myBuilderConstructor1 = LambdaMetafactory.metafactory(
            lookup,
            "apply",
            MethodType.methodType(MyBuilder1.class),
            mh.type().erase(),
            mh,
            mh.type()
        ).getTarget();

        MethodHandle myBuilderConstructor2 = LambdaMetafactory.metafactory(
            lookup,
            "apply",
            MethodType.methodType(MyBuilder2.class),
            mh.type().erase(),
            mh,
            mh.type()
        ).getTarget();

        @SuppressWarnings("unchecked")
        Function<String, IllegalArgumentException> functionFactory =
            (Function<String, IllegalArgumentException>) myFunctionConstructor.invokeExact();

        @SuppressWarnings("unchecked")
        MyBuilder1<IllegalArgumentException> myBuilder1Factory =
            (MyBuilder1<IllegalArgumentException>) myBuilderConstructor1.invokeExact();

        @SuppressWarnings("unchecked")
        MyBuilder2<IllegalArgumentException> myBuilder2Factory =
            (MyBuilder2<IllegalArgumentException>) myBuilderConstructor2.invokeExact();

        IllegalArgumentException runFunction = functionFactory.apply("test");
//      IllegalArgumentException runBuilder1 = myBuilder1Factory.apply("test");
        IllegalArgumentException runBuilder2 = myBuilder2Factory.apply("test");

    }
}

Why do runFunction and runBuilder2 work while runBuilder1 throws the following exception?

java.lang.AbstractMethodError: Receiver class Testcase$$Lambda$233/0x0000000800d21d88 does not define or inherit an implementation of the resolved method 'abstract java.lang.Object apply(java.lang.String)' of interface MyBuilder1.

Given that the IllegalArgumentException constructor takes a String parameter, not an Object, shouldn't the JVM accept runBuilder1 and complain about the parameter type of the other two?

Your MyBuilder1<R> has a functional method

R apply(String message);

whose erased type is

Object apply(String message);

In other words, unlike Function or MyBuilder2, the erased parameter type is String, rather than Object. The erase() method of MethodType just replaces all reference types with Object, which was handy for Function and MyBuilder2 but is not suitable for MyBuilder1 anymore. There is no similarly simple method for non-trivial types. You have to include type transformation code specifically for your case (unless you want to lookup the interface method via Reflection).

For example, we can just change the return type to Object and keep the parameter types:

class Testcase
{
    @FunctionalInterface
    public interface MyBuilder1<R>
    {
        R apply(String message);
    }

    public static void main(String[] args) throws Throwable
    {
        Class<?> clazz = IllegalArgumentException.class;

        MethodHandles.Lookup lookup = MethodHandles.lookup();
        MethodHandle mh = lookup.findConstructor(clazz,
            MethodType.methodType(void.class, String.class));

        MethodHandle myBuilderConstructor1 = LambdaMetafactory.metafactory(
            lookup,
            "apply",
            MethodType.methodType(MyBuilder1.class),
            mh.type().changeReturnType(Object.class), // instead of erase()
            mh,
            mh.type()
        ).getTarget();

        @SuppressWarnings("unchecked")
        MyBuilder1<IllegalArgumentException> myBuilder1Factory =
            (MyBuilder1<IllegalArgumentException>) myBuilderConstructor1.invokeExact();

        IllegalArgumentException runBuilder1 = myBuilder1Factory.apply("test");

        runBuilder1.printStackTrace();
    }

Regarding your last question, the erased type is the type to implement, whereas the last parameter to metafactory determines the intended type, i.e. derived from the Generic interface type. The generated code may have type casts from the erased type to this type when necessary. Since this type matches the constructor signature in all cases, all variants can invoke the constructor.