Doubly-boxed wildcard return type has an unexpected extra wildcard

I have a method that returns a doubly-boxed generic type. When that generic type is a wildcard, the return type is not the one that I expect.

An object of type Codec<T> can produce a Box<Box<T>>, so I would expect a Codec<?> to return a Box<Box<?>>, but instead it seems to return a Box<? extends Box<?>>. Where does that extra wildcard around the inner box come from ?

Minimal code for reproducing:

interface Codec<A>
{
    interface Box<B>{}

    public abstract Box<Box<A>> decode();

    static public <T> void SimpleDecode(Codec<T> codec){
        Box<Box<T>> result = codec.decode();
    }

    static public void ExpectedWildCardDecode(Codec<?> codec){
        Box<Box<?>> result = codec.decode();
        // Type mismatch: cannot convert from Codec.Box<Codec.Box<capture#1-of ?>> to Codec.Box<Codec.Box<?>>Java(16777233)
    }

    static public void ObservedWildcardDecode(Codec<?> codec){
        Box<? extends Box<?>> result = codec.decode();
        // Works, but why ?
    }
}

Codec, Box and decode are stand-ins for library classes and functions which I do not control.

Similar behaviour occurs with Codec<? extends T>.

I've already figured some workarounds for my use case, (sometimes using unchecked casts, sometimes using library-specific conversion methods), but I still want to understand why this did not work in the first place.

This was compiled for Java 21 with Gradle 8.11.1.

Solution

The type of the expression codec.decode() is not exactly Box<? extends Box<?>>. It is Box<Box<CAP#1>>, where CAP#1 is a fresh type variable. This just so happens to be convertible to Box<? extends Box<?>>, since Box<CAP#1> is a subtype of Box<?>.

This type is not denotable in Java, but you can say var result = codec.decode(); to have result actually be that type. Or you call a helper method like this:

static public void ObservedWildcardDecode(Codec<?> codec){
    helper(codec.decode());
}

static <T> void helper(Box<Box<T>> result) {
    // do things with 'result' here...
    // 'result' will have the same capabilities as 'Box<Box<CAP#1>>'
}

The type of the expression is not Box<Box<?>> because the return type of the method, Box<Box<?>>, undergoes capture conversion. This is a conversion that replaces all the wildcards in a type with fresh type variables with appropriate bounds (in this case, I have called the fresh type variable CAP#1). This is to capture (no pun intended) the idea that decode can return Box<Box<A>>, Box<Box<B>>, or Box<Box<C>>, or Box<Box<AnythingElse>>.

To see why this is needed, suppose decode returns a List<List<A>> instead (which is isomorphic to Box<Box<A>> from the type system's perspective). What can you do to this returned list? You cannot safely add a List<String> to this list of lists, because decode could have returned a List<Integer> instead.

List<List<?>> result = codec.decode(); // suppose this compiles
result.add(List.of(new Object())); // this could be unsafe if "codec.decode" actually returns a List<List<Integer>>!

// example implementation of Codec
class MyCodec implements Codec<Integer> {
    private List<List<Integer>> list = new ArrayList<>();

    @Override
    public List<List<Integer>> decode() {
        return list;
    }
    
    public void someOtherMethod() {
        // this would go very wrong if list.get(0).get(0) is actually just a 'new Object()'
        System.out.println(list.get(0).get(0) + 1);
    }
}