pythonpython-typingcovariancecontravariance

Python static type hint/check mismatch between Iterable[AnyStr] vs Iterable[str] | Iterable[bytes]


I'm running into this static type hint mismatch (with Pyright):

from __future__ import annotations
from typing import AnyStr, Iterable


def foo(i: Iterable[AnyStr]):
    return i


def bar(i: Iterable[str] | Iterable[bytes]):
    return i


def baz(i: Iterable[str | bytes]):
    return i


def main():
    s = ['a']

    # makes sense to me
    baz(foo(s))  # allowed
    foo(baz(s))  # not allowed

    # makes sense to me
    baz(bar(s))  # allowed
    bar(baz(s))  # not allowed

    bar(foo(s))  # allowed
    foo(bar(s))  # nope -- why?

What's the difference between Iterable[AnyStr] and Iterable[str] | Iterable[bytes]?

Shouldn't they be "equivalent"? (save for AnyStr referring to a single consistent type within a context)

More concretely: what is the right way to type-hint the following?

import random
from typing import Iterable, AnyStr

def foo(i: Iterable[AnyStr]):
    return i

def exclusive_bytes_or_str():  # type-inferred to be Iterator[bytes] | Iterator[str]
    if random.randrange(2) == 0:
        return iter([b'bytes'])
    else:
        return iter(['str'])

foo(iter([b'bytes']))          # fine
foo(iter(['str']))             # fine
foo(exclusive_bytes_or_str())  # same error

Solution

  • Paraphrased answer from erictraut@github:

    This isn't really the intended use for a constrained TypeVar. I recommend using an @overload instead:

    @overload
    def foo(i: Iterable[str]) -> Iterable[str]: ...
    @overload
    def foo(i: Iterable[bytes]) -> Iterable[bytes]: ...
    
    def foo(i: Iterable[AnyStr]) -> Iterable[AnyStr]:
        return i
    

    Because:

    The type Iterable[str] | Iterable[bytes] is not assignable to type Iterable[AnyStr]. A constrained type variable needs to be matched against one of its contraints, not multiple constraints. When a type variable is "solved", it needs to be replaced by another (typically concrete) type. If foo(bar(s)) were allowed, what type would the AnyType@foo type variable resolve to? If it were resolved to type str | bytes, then the concrete return type of foo would be Iterable[str | bytes]. That's clearly wrong.