pythongenericsmypypython-typing

Generic type-hinting for kwargs


I'm trying to wrap the signal class of blinker with one that enforces typing so that the arguments to send and connect get type-checked for each specific signal.

eg if I have a signal user_update which expects sender to be an instance of User and have exactly two kwargs: time: int, audit: str, I can sub-class Signal to enforce that like so:

class UserUpdateSignal(Signal):
  class Receiver(Protocol):
    def __call__(sender: User, /, time: int, audit: str):
      ...

  def send(sender: User, /, time: int, audit: str):
    # super call

  def connect(receiver: Receiver):
    # super call

which results in the desired behavior when type-checking:

user_update.send(user, time=34, audit="user_initiated") # OK

@user_update.connect # OK
def receiver(sender: User, /, time: int, audit: str):
    ...

user_update.send("sender") # typing error - signature mismatch

@user_update.connect # typing error - signature mismatch
def receiver(sender: str):
    ...

The issues with this approach are:

The ideal approach would apply a signature defined once to both send and connect - probably through generics. I've tried a few approaches so far:

Positional Args Only with ParamSpec

I can achieve the desired behavior using only

class TypedSignal(Generic[P], Signal):
    def send(self, *args: P.args, **kwargs: P.kwargs):
        super().send(*args, **kwargs)

    def connect(self, receiver: Callable[P, None]):
        return super().connect(receiver=receiver)


user_update = TypedSignal[[User, str]]()

This type-checks positional args correctly but has no support for kwargs due to the limitations of Callable. I need kwargs support since blinker uses kwargs for every arg past sender.

Other Attempts

Using TypeVar and TypeVarTuple

I can achieve type-hinting for the sender arg pretty simply using generics:

T = TypeVar("T")

class TypedSignal(Generic[T], Signal):
    def send(self, sender: Type[T], **kwargs):
        super(TypedSignal, self).send(sender)

    def connect(self, receiver: Callable[[Type[T], ...], None]) -> Callable:
        return super(TypedSignal, self).connect(receiver)

# used as
my_signal = TypedSignal[MyClass]()

what gets tricky is when I want to add type-checking for the kwargs. The approach I've been attempting to get working is using a variadic generic and Unpack like so:

T = TypeVar("T")
KW = TypeVarTuple("KW")

class TypedSignal(Generic[T, Unpack[KW]], Signal):
    def send(self, sender: Type[T], **kwargs: Unpack[Type[KW]]):
        super(TypedSignal, self).send(sender)

    def connect(self, receiver: Callable[[Type[T], Unpack[Type[KW]]], None]) -> Callable:
        return super(TypedSignal, self).connect(receiver)

but mypy complains: error: Unpack item in ** argument must be a TypedDict which seems odd because this error gets thrown even with no usage of the generic, let alone when a TypedDict is passed.

Using ParamSpec and Protocol

P = ParamSpec("P")

class TypedSignal(Generic[P], Signal):
    def send(self, *args: P.args, **kwargs: P.kwargs) -> None:
        super().send(*args, **kwargs)

    def connect(self, receiver: Callable[P, None]):
        return super().connect(receiver=receiver)


class Receiver(Protocol):
    def __call__(self, sender: MyClass) -> None:
        pass

update = TypedSignal[Receiver]()


@update.connect
def my_func(sender: MyClass) -> None:
    pass

update.send(MyClass())

but mypy seems to wrap the protocol, so it expects a function that takes the protocol, giving the following errors:

 error: Argument 1 to "connect" of "TypedSignal" has incompatible type "Callable[[MyClass], None]"; expected "Callable[[Receiver], None]"  [arg-type]
 error: Argument 1 to "send" of "TypedSignal" has incompatible type "MyClass"; expected "Receiver"  [arg-type]

Summary

Is there a simpler way to do this? Is this possible with current python typing?

mypy version is 1.9.0 - tried with earlier versions and it crashed completely.


Solution

  • After a lot of trial an error, I've found a relatively simple solution, although it depends on mypy_extensions which is deprecated so this may not be entirely future-proof, although it still works on the latest mypy version.

    Essentially, using mypy's NamedArg allows defining kwargs in a Callable, enabling us to simply use ParamSpec to solve this:

    class TypedSignal(Generic[P], Signal):
        def send(self, *args: P.args, **kwargs: P.kwargs):
            super().send(*args, **kwargs)
    
        def connect(self, receiver: Callable[P, None]):
            return super().connect(receiver=receiver)
    
    
    user_update = TypedSignal[[User, NamedArg(str, "metadata")]]()
    

    This correctly type-checks calls so that anything aside from the below:

    @user_update.connect
    def my_func(sender: User, metadata: str) -> None:
        pass
    
    user_update.send(User(), metadata="metadata")
    

    will throw an error.